🤖 AI

Specification-Based Code-Text-Code Reengineering for LLM-Mediated Software Evolution

This paper proposes a specification-based Code2Text2Code reengineering framework that mitigates semantic drift and behavioral inconsistencies in LLM-mediated software evolution by transforming source code into a neutral textual specification for iterative verification before regenerating the target code.

Original authors: Oleg Grynets, Vasyl Lyashkevych, Arsen Dolichnyi, Roman Piznak, Taras Zelenyy, Volodymyr Morozov

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Oleg Grynets, Vasyl Lyashkevych, Arsen Dolichnyi, Roman Piznak, Taras Zelenyy, Volodymyr Morozov

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to translate a complex, handwritten recipe from a grandparent's kitchen in 1950s Ukraine into a modern, digital recipe app for a smart oven in 2024.

If you just ask a translator (an AI) to "turn this old recipe into a new one," they might get the words right but mess up the cooking. They might keep using a "wooden spoon" instruction when the new oven needs a "digital sensor," or they might accidentally add a "secret ingredient" that wasn't in the original. The result looks like a recipe, but it might burn your cake.

This paper proposes a smarter way to do this translation, not just for recipes, but for computer software. They call it Code–Text–Code Reengineering.

Here is how it works, broken down into simple steps:

1. The Problem: The "Direct Translation" Trap

Usually, when people use AI to change software from one language to another (like moving from Python to Java), they do a Code-to-Code translation.

The Risk: The AI might copy the style of the old code instead of the meaning. It's like translating a poem word-for-word and losing the emotion. The new code might look correct but behave differently, or it might miss hidden rules that the old code relied on.

2. The Solution: The "Neutral Translator" (The Middleman)

Instead of jumping straight from Old Code to New Code, the authors insert a middle step: The Neutral Text Specification.

Think of this as a detailed, plain-English blueprint that describes what the software does, without mentioning how it was built in the old language.

Step 1 (Code to Text): The AI reads the old code and writes a neutral description. It says things like, "This function takes a list of numbers, adds them up, and saves the result to a database." It ignores the specific programming tricks used in the old code.
Step 2 (The Check): Humans or other tools check this description to make sure it matches the original perfectly. Did we miss a step? Did we invent a new one?
Step 3 (Text to Code): The AI reads this clean, neutral description and writes the new code in the target language. Because the AI isn't looking at the old code anymore, it doesn't accidentally copy the old style. It builds the new code exactly as the blueprint describes.

3. The "Safety Net" Tools

The paper explains that this isn't just about writing text; it's about building a safety system around the process:

The "Fact-Checker" (Retrieval): Before the AI writes anything, it looks up facts in a library of documents, diagrams, and code snippets to make sure it's not guessing. It's like a chef checking a cookbook before adding salt.
The "Chunking" Strategy: You can't read a whole library of books in one bite. The system breaks the code into small, logical "chunks" (like chapters) so the AI doesn't get confused or miss connections.
The "Blueprint" (Graphs & Ontology): The system builds a map (a graph) showing how different parts of the software connect. If the old software had a bridge between two islands, the new software must have that same bridge. This map helps measure if the translation was successful.

4. Measuring Success: Did We Lose Anything?

How do you know the new software is the same as the old one? The authors created a "Loss Calculator."
Imagine you are moving furniture from an old house to a new one.

Structural Preservation: Did you move all the rooms? (Yes/No)
Interface Stability: Can you still open the front door the same way? (Yes/No)
Total Similarity: A score that tells you how much of the original "house" survived the move.

5. What They Found

The researchers tested this method on many different types of code (SQL databases, web scripts, general programming).

The Result: Using the "Neutral Text" middleman worked better than trying to translate code directly. It reduced errors and made the new code behave more like the original intent.
The Catch: Sometimes, the AI needs a human to step in and clarify a confusing part of the old code. It's not fully automatic yet; it's a "human-in-the-loop" process.

The Big Picture

This paper argues that we shouldn't treat software updates as a simple "copy-paste" job. Instead, we should treat it as re-engineering.

Old Way: "Here is the old code; give me the new code." (Risky, opaque).
New Way: "Here is the old code. First, explain what it does in plain English. Check that explanation. Then, build the new code based on that explanation." (Safe, transparent, and controllable).

In short, they are teaching AI to stop translating words and start translating meaning, using a clear, neutral blueprint to ensure nothing gets lost in the shuffle.

Technical Summary: Specification-Based Code–Text–Code Reengineering for LLM-Mediated Software Evolution

1. Problem Statement

Large Language Models (LLMs) are increasingly utilized for software migration, refactoring, and code regeneration. However, direct Code-to-Code transformation approaches face significant limitations. These include:

Semantic Drift: Models may preserve surface-level syntax while altering underlying business logic or introducing hidden behavioral changes.
Non-Idiomatic Output: Generated code may retain source-language constructs that are inappropriate for the target environment.
Traceability Loss: There is often no clear link between the source code, the model's interpretation, and the regenerated artifact.
Contextual Gaps: Direct translation often fails to account for data dependencies, exception handling, and non-functional assumptions.

The paper argues that LLM-mediated software evolution should not be viewed merely as code generation but as a controlled reengineering process requiring intermediate representation layers to ensure semantic consistency and structural preservation.

2. Methodology: The Code–Text–Code Framework

The proposed framework introduces a Specification-Based Code–Text–Code reengineering model. The core transformation flow is defined as:
$C_0 \rightarrow S_n \rightarrow C_1$
Where $C_0$ is the source code, $S_n$ is a neutral textual specification, and $C_1$ is the regenerated target code.

A. The Neutral Textual Specification ( $S_n$ )

Unlike simple code summarization, $S_n$ is a structured, verifiable artifact designed to capture:

Program behavior, identifiers, and computational flow.
Conditions, side effects, and data dependencies.
Domain-specific intent without source-language syntax bias.
It serves as a "semantic mediator" separating what the software does from how it is implemented.

B. Knowledge Representation Layer

The framework integrates multiple representation layers to support traceability and validation:

AST & Metadata Extraction: Structured extraction of facts (function names, signatures, dependencies) to constrain the generation process and reduce hallucinations.
Ontology & Semantic Annotation: Mapping intermediate representations to structured knowledge (OWL/SKOS) to support reasoning and reuse.
Graph Representation: Utilizing Code Property Graphs (CPG) to track dependencies (calls, reads, writes) and estimate transformation losses.
Hybrid Retrieval: Combining vector search (for semantic similarity) with graph traversal (for structural dependencies) and metadata filtering to ground both Code2Text and Text2Code phases.

C. Pipeline Architecture

The end-to-end pipeline includes:

Factual Context Extraction: Gathering identifiers and dependencies before specification generation.
Code2Text Generation: Creating the initial neutral specification using RAG and factual context.
Iterative Validation & Refinement: Comparing the specification against source code to detect missing behaviors or excessive assumptions.
Human-in-the-Loop: Optional expert correction for ambiguous business logic or hidden assumptions.
Text2Code Generation: Generating target code based on the validated specification, ensuring idiomatic target-language implementation.
Target Verification: Validating syntax, functional equivalence, and interface stability.
DBMS Feedback (for SQL): Using execution-based validation to ensure dialect-specific correctness.

D. Semantic-Aware Chunking

The paper emphasizes that naive chunking breaks semantic units. It proposes an "ideal code chunk" defined by atomicity, self-parsability, and semantic awareness, evaluated via metrics like Boundary F1 and Chunk Certainty.

3. Key Contributions

The paper presents five primary contributions:

Conceptual Interpretation: Reframing Code–Text–Code not as a translation pipeline but as a specification-based reengineering process where the neutral text is a controlled knowledge artifact.
Pipeline Architecture: A comprehensive system integrating factual extraction, iterative validation, human correction, and DBMS-supported feedback.
Knowledge Representation Model: A unified model combining AST metadata, ontologies, graph dependencies, and neutral text to link source code, documentation, and regenerated artifacts.
Experimental Evaluation: A dataset covering multiple languages (Java, Python, C++, SQL dialects) and an evaluation protocol comparing intermediate representations (Pseudocode, Graph IR, Compiler IR, Natural Language IR).
Formal Loss Estimation: A graph-based formalization to quantify transformation losses using:
- Directional Structural Preservation ( $\alpha$ ): Measuring source dependency retention.
- Reverse Compatibility ( $\beta$ ): Measuring the introduction of unnecessary target dependencies.
- Interface Preservation ( $\delta_{IO}$ ): Ensuring input/output stability.
- Total Graph Similarity ( $S$ ): A weighted harmonic mean of the above.

4. Experimental Results

The authors conducted experiments using a dataset of 1,000 scripts per language across nine programming languages and five SQL dialects.

Intermediate Representation Comparison: Natural Language IR (the neutral specification) demonstrated the strongest practical suitability for LLM-mediated transformation compared to pseudocode, graph-based IR, or compiler-style IR.
Retrieval Evaluation: Vector retrieval proved more accurate for semantic relevance, while graph retrieval offered lower latency for dependency navigation. A hybrid approach was recommended.
Documentation Transformation: In TechDoc-to-BusinessDoc experiments, 22 out of 24 chunks passed validation. Failures involved logic distortion or scope omission, highlighting the need for validation loops.
Prompt Tuning: Using DSPy/MIPROv2, the authors found that the original, carefully constrained baseline prompt outperformed automatically generated variants. This suggests that for constrained reengineering tasks, explicit constraints (neutrality, structure) are more critical than prompt complexity.
Graph Metrics: The formal metrics successfully quantified structural preservation and interface stability, providing a method to detect "plausible but incorrect" transformations.

5. Significance and Claims

The paper claims that specification-based Code–Text–Code reengineering offers a controlled, traceable, and monitorable alternative to opaque direct code generation.

Transparency: By making the intermediate specification explicit, the model's interpretation becomes visible, reviewable, and correctable.
Quality Control: The framework moves beyond local syntactic correctness to evaluate semantic coverage, retrieval groundedness, and structural preservation.
Intelligent Monitoring: The authors position the framework not just as a transformation tool, but as an object of "intelligent monitoring," where semantic consistency and transformation losses are continuously assessed.
Reengineering vs. Translation: The work asserts that software evolution mediated by LLMs requires a shift from simple translation to a multi-level representation process involving code, technical documentation, business requirements, and graph structures.

The paper concludes that while the approach shows promise, it requires careful handling of chunking, retrieval grounding, and human-in-the-loop validation to mitigate risks of semantic drift and hallucination, particularly in high-stakes domains like financial or healthcare systems.