How AI and Reverse Engineering Are Rebuilding Each Other

In recent years, reverse engineering has seen remarkable progress, driven largely by advances in AI and machine learning. These technologies now play a key role in various tasks, such as code analysis, algorithm optimisation, data extraction, and pattern recognition.

But what’s even more fascinating is the two-way relationship between reverse engineering and AI. Not only can AI be used to carry out a wide range of reverse engineering tasks, but reverse engineering can also be used to train and improve AI systems and even develop new AI technologies. This feedback loop has driven significant advancements in both fields, pushing the boundaries of what we thought possible. Now, let’s explore these two directions through a few insightful examples.

AI-Driven Reverse Engineering: Transforming Traditional Tasks and Techniques

Here are some examples that illustrate how AI technologies are revolutionising the field of reverse engineering:

• Reproducing AI Systems through Reverse Engineering: AlphaZero—an AI that taught itself to master games like chess and Go through self-play and reinforcement learning—highlighted one of the most common challenges in scientific research; namely, reproducibility, especially when the original processes behind model training and design aren’t fully disclosed. Due to a lack of technical transparency, the open-source community launched Leela Zero (which later evolved into Leela Chess Zero), a project aimed at replicating AlphaZero’s success. Volunteers worked together to develop and train a neural network based on what was publicly known. Remarkably, the result rivalled AlphaZero’s performance, demonstrating the power of reverse engineering when key details are unavailable.
• Accelerating Malware Analysis with AI-Driven Automation: In cybersecurity, reverse engineering malware is often tedious and time-consuming. However, the integration of large language models (LLMs) with tools like IDA Pro and Ghidra has changed the game, as shown in this real-world application of LLMs in malware reverse engineering. These models were used to automate malware analysis and recognise patterns in malicious code, identifying Indicators of Compromise, classifying malware behaviour, and mapping threats within frameworks like MITRE ATT&CK. This level of automation reduced the manual effort while increasing speed and accuracy, offering significant potential to help analysts stay ahead of evolving cyber threats.
• Advancing Drug Discovery: In a case study published by the National Library of Medicine, researchers looked at how AI could support the reverse engineering of drug targeting in vivo contexts. By training AI models on multiple short molecular dynamics simulations, they were able to simplify the complex process of protein folding, enabling the propagation of folding trajectories beyond the limits of traditional methods. This allowed them to more accurately model how drugs interact with proteins inside the body—something that had previously been too complex to simulate accurately. By providing more accurate models of protein-drug interactions as they occur in vivo, AI could finally help close the gap between limited computational simulations and real, dynamic cellular environments, which is vital for effective drug development and optimisation.

Leveraging Reverse Engineering to Advance AI

The following case studies demonstrate how reverse engineering can be used to enhance AI technologies.

1. Reverse Engineering LLMs to Decode and Influence Model Behaviour: In a case study published in VOX, researchers from Anthropic (the AI company behind Claude) used reverse engineering to improve LLMs by analysing the activation patterns of neurons within the model. By probing the model and studying how neurons responded to various inputs, they identified millions of features, leading to concepts like “Golden Gate Bridge” and “Immunology”. Finally, manipulating these features allowed them to influence the model’s output according to the desired outcomes. This approach provides valuable insights into the inner workings of LLMs, allowing developers to improve their behaviour and contributing to the growing field of AI interpretability, ultimately making LLMs more effective.
2. ​Enhancing AI Security Through Reverse-Engineering Neural Networks Using Power Trace Analysis: This case study demonstrates how Deep Learning Side-Channel Analysis (DLSCA) can be used to reverse-engineer neural networks. Researchers implemented perceptron network models on an 8-bit microcontroller and captured power consumption traces during neural network execution. By using a trained MLP model to classify and analyse these traces, they were able to identify different activation functions and extract neural network weights, often with fewer than 1,000 traces. Their method offers valuable insight into neural network architectures and demonstrates how analysing power traces can reveal key information to strengthen AI security and drive further research in this field.
3. Reverse Engineering Teachable Machine’s Models to Enhance AI Model Development: Although not a formal case study, another example of reverse engineering being used to improve AI comes from a user of Google’s Teachable Machine—a platform that allows users to build deep learning models without coding. The process involved training an AI model, and then exporting and analysing its layers, configurations, and hyperparameters with an open-source tool called Netron. By reverse engineering the model, the user was able to understand its structure and gain insights into how similar AI models could be replicated and improved in future projects.
4. Reverse Engineering the Future of AI: To understand how reverse engineering could shape the next generation of AI and contribute not only to more advanced models but also to the broader pursuit of AGI and even ASI, there are two case studies worth exploring.
   1. In 2024, a group of researchers introduced DREAM, a framework designed to reverse-engineer black-box AI models without requiring access to their training data. By framing the challenge as one of out-of-distribution generalisation, DREAM used a meta-model to deduce specific characteristics, including the architecture and hyperparameters, solely through input-output observations. This approach enhanced the transparency and interpretability of current AI models, potentially paving the way for more accountable and explainable technologies—an essential step towards more advanced intelligent systems, such as AGI and ASI.
   2. The Department of Computer Science at the University of Sheffield has turned to nature for inspiration, reverse-engineering the brains of insects, particularly honeybees, to develop algorithms for robotic navigation and collision avoidance. Their method, which they called “natural intelligence,” moved away from traditional deep learning methods in favour of replicating the highly efficient neural processes found in insects. This biologically inspired reverse engineering approach offers a promising route to creating AI systems that are both energy-efficient and capable of sophisticated behaviours.

As AI continues to evolve, reverse engineering is set to play an increasingly crucial role in both understanding and shaping emerging intelligent systems. From replicating existing AI models to pushing towards more advanced forms of AI, the interplay between these two fields could determine how, and whether, we will ever reach AGI and, eventually, ASI—though both remain largely theoretical for now.