💻 computer science

Validated Code Translation for Projects with External Libraries

This paper presents a framework that enhances the translation of Go projects with external dependencies to Rust by combining an API retrieval mechanism with a cross-language validation pipeline that synthesizes adapters from public library APIs, thereby significantly improving compilation success rates and ensuring semantic equivalence for code manipulating opaque types.

Original authors: Hanliang Zhang, Arindam Sharma, Cristina David, Meng Wang, Brandon Paulsen, Daniel Kroening, Wenjia Ye, Taro Sekiyama

Published 2026-02-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hanliang Zhang, Arindam Sharma, Cristina David, Meng Wang, Brandon Paulsen, Daniel Kroening, Wenjia Ye, Taro Sekiyama

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive, complex library written in Go (a programming language), and you want to move all the books into a new, stricter, and safer library written in Rust. This is like moving a bustling city from one country to another where the laws, street signs, and even the language are slightly different.

The paper you shared describes a new "moving company" (a software tool) that uses AI to do this translation, but with a special twist: it solves two major problems that previous movers couldn't handle.

Here is the story of how they did it, using some everyday analogies.

The Problem: The "Hallucinating" Translator and the "Locked Boxes"

In the past, when AI tried to translate code, it faced two big headaches:

The Hallucinating Librarian:
Imagine you ask an AI to translate a Go function that uses a specific tool called sha512. The AI, trying to be helpful, might invent a fake Rust tool called sha512_rust_new that doesn't actually exist. Or, it might know the tool exists but forget to bring the "instruction manual" (the import statement) needed to use it.
- The Result: The translation looks good on paper, but when you try to build it, it crashes because the tools are missing or fake.
The Locked Boxes (Opaque Types):
In the old days, if you wanted to check if the translation worked, you'd just compare the "before" and "after" boxes. But in modern programming, libraries often use "opaque types."
- The Analogy: Imagine a Go program passes a locked safe to a function. The AI translates the code to Rust, but now it's a different kind of safe. You can't just open the safe to see if the contents are the same because the keys are different, and the safes are made of different materials. You can't even peek inside to compare the gold bars. Previous tools gave up here because they couldn't compare the "insides" of these locked boxes.

The Solution: The "CrossCrate" Moving Company

The authors built a system called CrossCrate that acts like a super-smart, well-prepared moving team. They solved the problems in two main stages:

Stage 1: The "Real-Time Dictionary" (RAG)

To stop the AI from hallucinating fake tools, they gave it a Real-Time Dictionary (technically called Retrieval-Augmented Generation or RAG).

How it works: Before the AI tries to translate a line of code, it stops and asks a giant, verified database: "Hey, what is the exact Rust equivalent of this Go tool? And what is the exact line of code I need to type to make it work?"
The Analogy: Instead of letting the AI guess from memory (which is prone to errors), they hand it a verified map of the Rust city. If the Go code uses a specific library, the map tells the AI exactly which Rust crate (library) to use and, crucially, exactly which "import" lines to write so the code actually compiles.
The Result: The AI stops making up fake tools. It only uses real, existing tools with the correct instructions.

Stage 2: The "Universal Packing Crate" (The Adapter)

To solve the "Locked Box" problem, they invented a Universal Packing Crate.

The Problem: You can't compare a Go "safe" directly to a Rust "safe."
The Solution: They created a neutral, standard container (based on a format called Protobuf).
1. Packing: They take the Go "safe," open it (using the library's public keys), and pack its contents into this Universal Crate.
2. Unpacking: They take the Rust "safe," open it, and pack its contents into an identical Universal Crate.
3. Comparison: Now, instead of comparing two different safes, they just compare the Universal Crates. Since the crates are identical by design, if the contents match, the translation is correct!
The Analogy: Imagine you are moving a fragile vase. You can't compare the vase in the old house to the vase in the new house directly because they are wrapped differently. So, you take the vase out, put it in a standard shipping box, move it, and then check if the box contents match. If they do, the vase is safe.

The Results: A Perfect Move

The team tested this on six real-world, complex Go projects (like password generators and encryption tools) that relied heavily on outside libraries.

Without their help: The AI failed to compile the code or couldn't prove it worked (0% success on the hardest tasks).
With their help: They achieved a 95% to 100% success rate.
- The code actually compiled (no fake tools).
- The code behaved exactly the same (the "Universal Crates" matched).

The Takeaway

This paper is about teaching AI to be a careful, well-researched translator rather than a confident guesser.

Don't guess the tools: Use a verified database to find the right tools and instructions.
Don't peek inside the locked boxes: Use a standard "packing crate" to move data between languages so you can compare them fairly.

By doing this, they made it possible to automatically and safely migrate complex, real-world software from one language to another, even when that software relies on mysterious, locked-down libraries.

1. Problem Statement

The paper addresses the critical limitations of using Large Language Models (LLMs) for translating software projects (specifically Go to Rust) that rely heavily on external libraries. Existing approaches fail in two primary areas:

Hallucination and Import Resolution: LLMs frequently hallucinate non-existent target APIs or select incorrect libraries. Furthermore, they often fail to generate the necessary import statements required to make those APIs callable in Rust, particularly when APIs are re-exported via traits or crates.
Validation of Opaque Types: Validating semantic equivalence between source and target code is difficult when functions manipulate opaque library-defined types (types with private fields or complex internal invariants). Traditional differential testing assumes structural identity of input/output types, which breaks down when types are defined in external libraries and cannot be directly compared or serialized across language boundaries.

2. Methodology

The authors propose CrossCrate, a translation and validation framework that combines Retrieval-Augmented Generation (RAG) with a schema-backed cross-language interoperability pipeline. The approach consists of three main stages:

Stage 1: Cross-Language Library Mapping (RAG)

To prevent hallucinations and resolve imports, the system augments the LLM prompt with a Crate-Scoped Knowledge Base:

Crate Matching: Maps Go packages to corresponding Rust crates (e.g., crypto/sha512 $\to$ sha2).
Knowledge Base Construction ( $K_{crate}$ ): For each target crate, the system indexes public APIs, their documentation, and crucially, their valid import paths (handling re-exports and trait scopes).
Retrieval: When translating a Go function, the system queries $K_{crate}$ using the Go API's name and documentation to retrieve the top-ranked Rust API candidates along with the exact import statements required to use them.

Stage 2: Translation Generation

The LLM generates the Rust translation using the retrieved API candidates and import paths as context. The translation is modular, handling types and functions in isolation, guided by a dependency summary.

Stage 3: Cross-Language Validation (The Core Innovation)

To validate equivalence without direct memory sharing or structural type matching, the authors introduce a Schema-Backed Carrier Representation:

Protobuf Schema Synthesis: An LLM synthesizes a language-neutral Protobuf schema ( $T_{Proto}$ ) from the source Go types. This schema acts as an intermediate data format (IDF).
Carrier Types: The Protobuf compiler generates concrete carrier types in both Go ( $T_{Go}^{Proto}$ ) and Rust ( $T_{Rust}^{Proto}$ ).
Adapter Synthesis: The system synthesizes "glue code" (adapters) to convert between:
- Original Go types $\leftrightarrow$ Go Carrier ( $T_{Go}^{Proto}$ )
- Original Rust types $\leftrightarrow$ Rust Carrier ( $T_{Rust}^{Proto}$ )
- Constraint: These adapters are synthesized exclusively using public library APIs (constructors, serialization routines) rather than direct field access, ensuring they respect library invariants.
Round-Trip Validation: Before testing equivalence, the system verifies that the adapters preserve information by checking round-trip properties (e.g., $A^{-1}_{Go}(A_{Go}(v)) = v$ ) on unit-test inputs.
Differential I/O Testing: Once interoperability is established, the system runs the original Go function and the translated Rust function on the same inputs (converted via the carrier). It compares the outputs after converting them back to a common representation to verify I/O Equivalence.

3. Key Contributions

First Validated Translation for Opaque Types: The first approach capable of validating the equivalence of functions with structurally dissimilar and opaque input/output types defined in external libraries.
RAG for Import Resolution: A novel retrieval mechanism that not only maps APIs but also resolves the specific, non-local import paths required for Rust compilation, significantly reducing hallucination.
Schema-Backed Interoperability: A method to bridge Go and Rust using a Protobuf intermediate format and synthesized adapters, enabling differential testing without requiring shared runtimes or direct memory access.
Comprehensive Evaluation: A framework that validates both compilation success and functional equivalence, moving beyond simple syntax checking.

4. Experimental Results

The system was evaluated on six real-world Go repositories with non-trivial external dependencies (including cryptographic libraries like age, pbkdf2, and argon2).

Overall Success Rate: The approach achieved an average 95.83% success rate for both compilation and I/O equivalence across all benchmarks.
Impact of RAG:
- Full System: 95.83% (Comp) / 95.83% (Equiv).
- No RAG (Baseline): 41.33% (Comp) / 0% (Equiv).
- No Imports: 41.33% (Comp) / 0% (Equiv).
- Observation: Without the RAG-based import resolution, the LLM fails to produce compilable code for library-dependent functions, and without correct imports, equivalence validation is impossible.
Comparison to Baseline: Compared to Oxidizer (a state-of-the-art tool), CrossCrate showed a 2x improvement on average, and up to 100% improvement in the most dependency-heavy case (age), where Oxidizer achieved 0% success on library-dependent functions.

5. Significance

This work represents a significant step forward in the automation of systems code migration (e.g., Go to Rust). By solving the "opaque type" problem, it enables the safe translation of complex, production-grade systems that rely on third-party libraries, which were previously considered too risky to automate. The methodology of using a language-neutral carrier (Protobuf) combined with public-API-only adapters provides a robust pattern for cross-language verification that can be applied to other language pairs beyond Go and Rust.