🤖 machine learning

Generative Compilation: On-the-Fly Compiler Feedback as AI Generates Code

This paper introduces "generative compilation," a novel approach that utilizes a lightweight "sealor" transformation to enable standard compilers to provide real-time feedback on partial programs during AI code generation, thereby reducing error cascades and improving functional correctness for complex languages like Rust without requiring white-box model access.

Original authors: Niels Mündler-Sasahara, Hristo Venev, Dawn Song, Martin Vechev, Jingxuan He

Published 2026-07-16

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Niels Mündler-Sasahara, Hristo Venev, Dawn Song, Martin Vechev, Jingxuan He

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a very talented, but slightly impulsive, robot to write a story. This robot, a Large Language Model (LLM), is great at guessing the next word in a sentence, but it doesn't actually know the rules of grammar or logic. It might write a sentence that sounds poetic but breaks the laws of physics, or in the world of coding, it might write a program that looks cool but crashes the computer. To fix this, we usually let the robot finish the whole story, then hand it to a strict editor (a compiler) who checks the rules. If the editor finds a mistake, they send the whole story back with a red pen, and the robot has to start over or try to fix the mess. The problem is, by the time the editor speaks, the robot might have written fifty more sentences based on that one tiny mistake, creating a huge, tangled mess that is hard to untangle.

This paper explores a new way to help the robot: instead of waiting until the end, what if the editor could whisper corrections while the robot is still writing? The authors, a team of computer scientists, call this "Generative Compilation." They figured out a clever trick to let standard, off-the-shelf code editors (compilers) check the robot's work as it happens, without needing to rebuild the editor from scratch or peek inside the robot's brain. They tested this on the programming language Rust, which is famous for being very strict about safety but also very hard to get right. Their findings suggest that by catching mistakes early, the robot makes fewer errors, writes better code, and finishes its tasks faster, all without needing a super-expensive, custom-built editor.

The Problem: The "Too Late" Editor

Think of writing code like building a house with a very strict architect. In the old way (called "Post-Generation Feedback"), the builder (the AI) constructs the entire house, brick by brick, without asking for help. Once the roof is on, the architect walks in, sees that the foundation is cracked, and says, "Start over." The builder has to tear down the whole thing, even though the mistake happened in the first brick. This is wasteful and frustrating.

Another method, called "Constrained Decoding," tries to stop the builder from laying a bad brick in the first place. But this requires the builder to be transparent (like a glass house) so the architect can see exactly what the builder is thinking and stop them before they make a move. Most of the best AI builders today are "black boxes"—we can't see inside their heads, so we can't use this method. Plus, building a custom architect who understands every possible rule of a complex language like Rust is incredibly difficult and expensive.

The Solution: The "Seal-or" Trick

The authors introduce a middle ground called Generative Compilation. The secret weapon here is a tool they call a Seal-or (a play on the word "seal").

Imagine the AI is writing a letter, but it stops in the middle of a sentence: "The cat sat on the..."
A normal compiler (the strict editor) would look at this incomplete sentence and say, "I can't grade this; it's not a real sentence."
The Seal-or is a clever translator. It takes that incomplete sentence and instantly fills in the blanks with "dummy" words that make the sentence grammatically complete, just for a split second. It turns "The cat sat on the..." into "The cat sat on the [magic placeholder]."

Now, the strict compiler can look at this "sealed" sentence. If the grammar is wrong (e.g., the cat can't sit on a magic placeholder), the compiler yells, "Hey, the cat can't sit on that!" The Seal-or then translates that complaint back to the AI: "You can't put the cat on the placeholder; try a different word." The AI then fixes the mistake immediately, before it writes the next sentence.

The magic of the Seal-or is that it is designed to be very careful. It only fills in the blanks with things that could be correct. If the AI's partial sentence could eventually become a perfect sentence, the Seal-or makes sure the compiler doesn't reject it just because it's incomplete. But if the AI has made a fatal mistake (like a "borrowing" error in Rust, where a variable is used in two places at once), the Seal-or ensures the compiler catches it right away.

What They Found: Catching Mistakes Early

The team tested this idea on real-world coding tasks, asking seven different AI models to write Rust programs. They compared three methods:

The AI alone: Just guessing.
The AI + Post-Generation Feedback: The AI writes the whole thing, gets corrected, and tries again.
The AI + Generative Compilation: The Seal-or checks the work while the AI writes.

The results were impressive. When using Generative Compilation:

Fewer Crashes: The number of programs that failed to compile dropped significantly. For some models, the error rate went from over 60% down to single digits.
Better Code: The programs that did run worked correctly more often.
Faster Fixes: The AI spent less time generating garbage code. Because the Seal-or caught errors early (often just a few lines after the mistake happened), the AI didn't waste time writing a whole chapter of code based on a wrong idea.
Smaller Complaints: Instead of getting a massive list of 20 errors at the end, the AI got small, focused hints about the specific mistake, making it easier to understand what went wrong.

Why This Matters

This approach is a big deal because it doesn't require us to rebuild the AI or the compiler. It works with the "black box" models we already have and uses the standard compilers that programmers already trust. It turns the compiler from a "final judge" into a "live coach."

The authors proved mathematically (using a formal language called Lean) that their Seal-or is safe: it won't trick the compiler into accepting bad code, and it won't reject code that could have been fixed. They then built a working version for the real Rust language and showed it works in practice.

In short, Generative Compilation is like giving the AI a safety net that catches it before it falls, rather than waiting until it hits the ground to tell it it fell. It makes AI coding safer, faster, and more reliable, especially for languages that demand high precision.

Technical Summary: Generative Compilation

Problem Statement

Large Language Models (LLMs) have become standard tools for code generation, yet the code they produce is often error-prone, particularly in languages with rich static semantics like Rust. While Rust's ownership and borrowing discipline provides strong memory safety guarantees, these strict compile-time rules make valid code generation difficult.

Existing strategies to address this gap suffer from significant limitations:

Post-Generation Compiler Feedback: The standard approach waits until a full program is generated, runs the compiler, and feeds diagnostics back to the model. This introduces a "delayed feedback" loop. Errors made early in generation can snowball, causing the model to waste tokens on invalid assumptions before receiving any correction. Furthermore, accumulated diagnostics for an entire file can be overwhelming, hindering the model's ability to focus on the root cause.
Constrained Decoding: This approach intervenes during token sampling, rejecting tokens that cannot lead to a valid program. While this guarantees validity upon termination, it requires white-box access to the model's distribution (ruling out frontier black-box APIs) and necessitates a costly, language-specific reimplementation of the compiler's semantic analysis (e.g., re-implementing Rust's borrow checker). For expressive languages, this is often impractical.

Methodology: Generative Compilation

The authors introduce Generative Compilation, a framework that enables off-the-shelf compilers to check partial programs during generation, providing compiler-style diagnostics without modifying the LLM's sampling process.

Core Concept: The Sealor

The central technical device is the sealor, a lightweight, mostly syntax-guided transformation $S$ that converts a partial program (a prefix of code) into a complete, well-formed program that a standard compiler can diagnose.

Mechanism: The sealor fills in missing syntactic structures and inserts well-typed placeholders (e.g., holeval() for values, holediv() for diverging control flow) to close the partial program.
Feedback Loop: The sealed program is passed to the existing compiler (e.g., rustc). If the compiler rejects the sealed program, the resulting diagnostic is mapped back to the original partial program's source spans and fed to the LLM. The LLM then revises the code based on this focused feedback.
Design Goals: The sealor is designed to satisfy two properties:
- Completeness: It must never reject a partial program that admits a valid completion (avoiding false positives that confuse the model).
- Soundness (Selective): It should reject partial programs that are genuine dead ends. While global soundness is undecidable for expressive semantics, the authors target soundness on specific classes of programs (e.g., at statement boundaries) to catch errors early.

Formalization and Implementation

Featherweight Rust (FR): The authors first formalize the approach on FR, a compact calculus capturing Rust's ownership and borrowing. They define a syntax-guided sealor ( $S_{FR}$ ) and prove its global completeness and selective soundness. This entire development, including the proofs, is mechanized in Lean.
Real Rust: The methodology is extended to real Rust ( $S_{RS}$ $S_{R S}$ ). The sealor handles Rust-specific syntax, control flow, and future-dependent errors (e.g., unresolved names that might be defined later).
- Placeholders: It uses holediv() (panic!) to close diverging branches without triggering borrow-check errors, and holeval() (a generic function call) to provide values without imposing specific type constraints.
- Error Suppression: To maintain completeness, the system suppresses specific compiler errors (e.g., "missing trait items") that arise solely from incomplete code but would be resolved by future generation.
- Projection: Diagnostics from the sealed program are projected back to the original partial program's source spans to ensure the LLM receives relevant feedback.

The system operates as a concurrent module alongside the LLM, streaming partial code to the sealor/compiler and restarting generation upon rejection, communicating only via plain text to support black-box models.

Key Contributions

Generative Compilation Framework: A novel framework enabling standard compilers to check partial programs during generation, bridging the gap between post-generation feedback and constrained decoding.
Sealor Theory: The definition of sealors and formal proofs that their completeness and soundness properties lift to the induced generative compiler.
Mechanized FR Implementation: A fully mechanized implementation in Lean of a syntax-guided sealor for Featherweight Rust, including corrections to the original FR formalization.
First Partial-Program Checker for Rust: A practical implementation for real Rust that delegates semantic reasoning to rustc while handling syntax, control flow, and placeholders.
Empirical Evaluation: Extensive evaluation across seven frontier black-box and open-weight models on repository-level Rust tasks (C-to-Rust translation and API adaptation).

Results

The evaluation demonstrates that generative compilation (GC) outperforms standard post-generation compiler feedback (PC) and raw LLM generation:

Reduced Compiler Errors: GC significantly reduces the rate of non-compiling outputs. For example, on the UpdatedAPI task, GC reduced compiler errors to 0.0% for Claude Opus 4.8 (compared to 6.7% with PC) and 3.3% for GPT-5 (compared to 13.3% with PC).
Improved Functional Correctness: GC achieved the highest functional correctness in 11 out of 14 model-dataset configurations.
Early Error Detection: GC detects errors much closer to their source. The median delay in reporting an error was only 3 lines after the error's primary span, compared to 14 lines for function-level checks and 89 lines for post-generation feedback.
Focused Diagnostics: By interrupting generation at the first unrecoverable error, GC produces smaller, more targeted error reports (average 5.5 diagnostics vs. 13.8 for PC), preventing error cascades.
Runtime Efficiency: Despite the overhead of on-the-fly checking, GC reduced overall runtime for models requiring many feedback rounds by avoiding the generation of long, non-compilable files.

Significance and Claims

The paper claims that generative compilation represents a step toward making compilers "first-class citizens" of AI-assisted programming, active during generation rather than as a separate post-hoc check.

Practicality: Unlike constrained decoding, it works with black-box models and reuses existing compiler infrastructure, avoiding the need for massive semantic reimplementation.
Effectiveness: It provides early, focused feedback that prevents the "snowball" effect of errors, leading to higher quality code and faster convergence.
Generality: The concept is applicable to any code generation method steerable by natural language prompts, not just LLMs.

The authors remain modest regarding the formalization of error messages, noting that while they suppress sealing artifacts empirically, formally proving that diagnostics correspond to genuine defects in the original partial program remains future work. They also acknowledge that automating the construction of sealors from language specifications is an open direction for future research.