🤖 AI

Decaf: Improving Neural Decompilation with Automatic Feedback and Search

The paper introduces Decaf, a system that leverages compiler feedback and search to significantly improve the semantic correctness of neural decompilation outputs, raising the success rate on the Real -O2 split from 26.0% to 83.9% without compromising similarity to the original source code.

Original authors: Alexander Shypula, Osbert Bastani, Edward Schwartz

Published 2026-05-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Alexander Shypula, Osbert Bastani, Edward Schwartz

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Lost Translation"

Imagine you have a book written in a complex, high-level language (like English). A machine translates this book into a secret code (machine code) to run on a computer. Once the translation happens, the machine throws away the original book, the chapter titles, the character names, and even the grammar rules.

Now, imagine you are a detective trying to figure out what the original story was, but you only have the secret code. This is decompilation.

Traditional tools (like Ghidra) are like a very literal translator. They can turn the secret code back into words, but the result is a mess. The sentences are awkward, the names are replaced with "Variable_1" and "Variable_2," and the logic is hard to follow. It's technically correct, but it's impossible to read.

On the other hand, modern AI (Large Language Models) is like a creative writer. It can guess the story, invent cool character names, and write smooth sentences. But because it's so creative, it sometimes hallucinates. It might invent a plot twist that never happened or miss a crucial detail, making the story factually wrong even if it reads beautifully.

The Solution: "Decaf" (Decompilation with Automated Feedback)

The authors of this paper, Alexander Shypula and his team, realized that relying on just one guess from the AI isn't enough. They built a system called Decaf that works like a talent show with a strict judge.

Here is how the Decaf process works, step-by-step:

1. The "Talent Show" (Sampling Many Candidates)

Instead of asking the AI to write the story once and hoping for the best, Decaf asks the AI to write 32 different versions of the story.

Analogy: Imagine asking 32 different chefs to cook the same dish. Some might burn it, some might make it too salty, but one of them might just happen to make the perfect version.
The paper found that if you only ask for one dish, you have a 60% chance of getting something edible. If you ask for 32, you have an 88% chance that at least one of them is perfect.

2. The "Taste Test" (Automatic Feedback)

Now you have 32 different versions of the code. How do you know which one is the real original? You can't just read them; they all look like code.

The Trick: Decaf takes every single AI-generated version and re-compiles it. It turns the code back into the secret machine code.
The Comparison: It then compares this new secret code against the original secret code you started with.
Analogy: Imagine you have the original secret recipe. You take the 32 chefs' dishes, turn them back into ingredients, and see which set of ingredients matches the original list exactly. If the ingredients match, the dish is correct.

3. The "Head Judge" (The Neural Reranker)

Sometimes, the re-compiling step isn't enough because the ingredients might look slightly different but taste the same. So, Decaf uses a second AI, called a Reranker, to act as the Head Judge.

This judge looks at the "secret code" of the original and the "secret code" of the AI's guess.
It doesn't just look at the words; it looks at the logic. It asks, "Do these two pieces of code do the exact same thing?"
The judge picks the winner and discards the rest.

The Results: Why It Matters

The paper tested this system on a massive benchmark called ExeBench. Here is what happened:

Before Decaf: The best AI models could only get the logic right about 26% of the time. They were either too messy (like traditional tools) or too creative (hallucinating errors).
With Decaf: The system jumped to 83.9% accuracy.
The "Perfect Match": Even better, the code produced by Decaf was so similar to the original that, in 70.9% of cases, if you re-compiled it, the computer code was byte-for-byte identical to the original.

A Real-World Example from the Paper

The paper shows a specific function (a small piece of code) that calculates numbers.

Traditional Tool (Ghidra): Gave a correct but ugly answer with names like iVar1 and iVar2.
Standard AI (LLM4Decompile): Gave a beautiful, readable answer, but it missed a critical step (a "break" condition), making the logic wrong.
Decaf: Generated 32 versions. It found the one that had the beautiful names and the correct logic. It successfully picked the winner using its "taste test" and "Head Judge."

The "Stress Test"

The authors also tested if their system worked when the "ingredients" changed. They tried to use a different compiler (Clang instead of GCC) to check the answers.

Result: The system still worked well, though it got slightly less accurate. This is like a judge who is used to tasting Italian food but is asked to judge French food; they can still tell if the dish is good, but they aren't quite as perfect as when judging their native cuisine.

Summary

Decaf doesn't try to make the AI smarter by feeding it more data. Instead, it changes the strategy:

Generate many options (don't settle for the first guess).
Verify them automatically by turning them back into machine code.
Use a smart judge to pick the one that is both readable and factually correct.

This approach turns a "guessing game" into a "search and verify" process, dramatically improving the ability to understand computer code that has been compiled and stripped of its original meaning.

Technical Summary: DECAF - Improving Neural Decompilation with Automatic Feedback and Search

1. Problem Statement

Decompilation is the process of translating low-level assembly code back into high-level source code. While traditional decompilers (e.g., Ghidra, Hex-Rays) employ sophisticated program analysis to recover variables and control flow, they often fail to reconstruct idiomatic syntax, meaningful identifier names, and custom data types. Conversely, recent neural decompilation models, particularly those based on Large Language Models (LLMs), excel at generating readable, idiomatic code but suffer from "hallucinations." These models frequently produce semantically incorrect code, omit critical logic (such as break conditions), or generate code that fails to compile or execute correctly.

The core challenge addressed by this paper is the trade-off between idiomaticity (readability and structure) and functional correctness (semantic equivalence to the original binary). Existing neural approaches typically generate a single candidate output, leaving the potential for correct solutions untapped if the model's first guess is flawed. Furthermore, verifying the correctness of decompiled code is an undecidable problem in the general case, making it difficult to automatically select the best candidate from a set of possibilities.

2. Methodology

The authors propose DECAF (DECompilation with Automated Feedback), a system that improves neural decompilation by generating multiple candidate decompilations and automatically selecting the most promising one using compiler feedback and a trained neural reranker.

2.1. The DECAF Pipeline

The system operates in three primary stages:

Initial Decompilation: A traditional decompiler (Ghidra) processes the target binary to produce an initial, often non-idiomatic, C-like representation.
Candidate Generation: This output is fed into a neural generator model (an LLM). Instead of a single inference, the system uses temperature sampling to generate a diverse set of $N$ candidate decompilations (e.g., $N=32$ ).
Automatic Feedback and Reranking:
- Compilation Filter: Each candidate is compiled (e.g., using gcc -c). Candidates that fail to compile are discarded.
- Reranking: The remaining candidates are evaluated against the original binary. The authors propose three reranking strategies:
  - Log Probability: Ranking based on the generator model's normalized log probability.
  - Byte Distance: Compiling the candidate and computing a modified Levenshtein distance between the candidate's binary bytes and the original binary's bytes. This distance accounts for relocation artifacts by treating specific offsets as wildcards.
  - Neural Reranker: A dedicated LLM (Decaf-ReRanker) trained to predict the likelihood that two assembly sequences (the original and the recompiled candidate) are functionally equivalent.

2.2. Data and Training

Generator Model: The authors fine-tuned LLM4Decompile-Ref-22b-v2 (22B parameters) on the EXEBENCH dataset. They introduced type-aware supervision, training the model to generate necessary #include statements, forward declarations, and type definitions to ensure the output is compilable. They also trained models on stripped binaries to simulate real-world reverse engineering scenarios.
Reranker Model: A 32-billion parameter model (QWEN3-32B) was trained to distinguish between functionally equivalent and non-equivalent assembly pairs. The training data was constructed by generating multiple candidates, executing them against test cases, and pairing successful executions (positive) with failures or mismatches (negative).
Vulnerability Recovery: For specific experiments, the authors utilized the Juliet Test Suite to evaluate the system's ability to recover code containing known vulnerabilities (CWE classes) from stripped binaries.

3. Key Contributions

DECAF System: A novel framework that combines neural generation with compiler feedback and search (sampling multiple candidates) to resolve the trade-off between readability and correctness.
Decaf-Gen-22b: A new LLM generator model that outperforms existing state-of-the-art neural decompilers (including LLM4Decompile and Idioms-Gemma-7b) on functional accuracy and byte-wise matching, particularly when trained with type-aware supervision and stripped binary inputs.
Decaf-ReRanker-32b: A specialized reranker model that effectively identifies functionally correct decompilations from a pool of candidates, significantly outperforming simple log-probability or byte-distance heuristics in many scenarios.
Methodological Insight: The demonstration that sampling multiple candidates (e.g., 32) dramatically increases the probability of finding a functionally correct solution (raising the upper limit from ~60% to ~88% in some splits), provided an effective reranking mechanism is in place.
Generalizability: Evidence that the reranking methodology can amplify weaker models (e.g., applying the DECAF reranker to LLM4Decompile outputs yields significant gains).

4. Experimental Results

The authors evaluated their approach on the EXEBENCH dataset across various splits (Real/Synth, Stripped/Unstripped, -O0/-O2).

Functional Correctness: On the Stripped REAL -O2 split, DECAF achieved an 83.9% functional correctness rate with reranking, compared to 26.0% for the best prior neural model (Idioms-Gemma-7b) and 59.1% for the generator alone (single sample).
Byte-Wise Match: DECAF achieved a 70.9% exact byte-wise match on the same split, significantly outperforming the 19.1% of Idioms-Gemma-7b.
Similarity to Source: Contrary to the intuition that optimizing for correctness might reduce readability, the authors found that reranking for functional correctness generally improved the similarity (lower Edit Distance) to the original source code.
Vulnerability Recovery: On the Juliet test suite, DECAF achieved an F1 score of 29.8% for vulnerability recovery, substantially outperforming Ghidra (1.7%) and demonstrating high precision (95.5%).
Robustness: When tested with a different compiler (Clang) for feedback than the one used for generation (GCC), the neural reranker's performance dropped on unstripped splits due to distribution shift, highlighting a dependency on training data diversity.

5. Significance and Claims

The paper argues that the limitations of current neural decompilers are not solely due to a lack of data or model size, but rather the lack of verification and search. By treating decompilation as a search problem where multiple candidates are generated and filtered via compiler feedback, the system can recover high-quality source code that is both readable and semantically correct.

The authors claim that DECAF represents a significant advancement in the state-of-the-art for neural decompilation, moving beyond the "best single guess" paradigm. They emphasize that their approach is general and can be applied to amplify any neural decompiler capable of producing diverse candidates. The work also highlights the practical utility of these methods in security tasks, such as recovering vulnerabilities from stripped binaries, where traditional tools often fail to provide actionable, high-level code.

The authors acknowledge limitations, including the reliance on access to the reference compiler configuration for byte-wise matching and the challenges of generalizing to different compiler architectures (e.g., Clang vs. GCC) without specific training. They conclude that future work should focus on training rerankers on more diverse compiler configurations and exploring reinforcement learning for iterative refinement.