💻 computer science

MR-Adopt: Automatic Deduction of Input Transformation Function for Metamorphic Testing

MR-Adopt is a novel approach that leverages large language models and data-flow analysis to automatically deduce generalizable input transformation functions from hard-coded metamorphic relations, thereby enabling their reuse with new inputs and significantly improving test adequacy.

Original authors: Congying Xu, Songqiang Chen, Jiarong Wu, Shing-Chi Cheung, Valerio Terragni, Hengcheng Zhu, Jialun Cao

Published 2026-04-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Congying Xu, Songqiang Chen, Jiarong Wu, Shing-Chi Cheung, Valerio Terragni, Hengcheng Zhu, Jialun Cao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a quality inspector for a factory that makes custom cakes. You have a recipe book (the software) and a set of test notes written by the original bakers (the developers).

The Problem: The "Hard-Coded" Recipe

Most of the time, these test notes are incredibly specific but not very useful for new situations. They look like this:

"Take a cake made on January 1st. Make a second cake for January 2nd. Check if the second cake is exactly one day older than the first."

This works great for testing those two specific dates. But what if you want to test the factory on February 29th or December 31st? The notes don't tell you how to get from the first date to the second. They just say, "Here is Date A, and here is Date B."

Because the notes don't explain the rule (the transformation), you can't automatically generate new tests. You are stuck manually writing new notes for every single date you want to test. This is a huge bottleneck. In the world of software, researchers found that 70% of these test notes are "hard-coded" like this—they have the inputs and the expected result, but they are missing the "magic formula" that connects them.

The Solution: MR-Adopt (The "Rule Detective")

The paper introduces a new tool called MR-Adopt. Think of MR-Adopt as a super-smart detective (powered by Large Language Models, or AI) whose job is to look at those specific, hard-coded notes and figure out the hidden rule.

Here is how MR-Adopt works, using a simple analogy:

Phase 1: The "Imagination Game" (Generating Examples)

If you only show the detective one example (Jan 1 $\to$ Jan 2), they might guess the rule is "Add 1 day." But they might also guess the rule is "Change the year to 2025" because that's the only other thing that changed.

To stop the detective from guessing wrong, MR-Adopt asks the AI to imagine more examples first.

"Okay, if Jan 1 becomes Jan 2, what would happen if we started with June 15? Would it become June 16?"
"What if we started with February 28? Would it become March 1?"

By generating many of these "what-if" scenarios, the AI gets a better sense of the pattern rather than just copying the specific numbers. It's like showing a child a few examples of "adding one" before asking them to solve a new math problem, so they don't just memorize the answer.

Phase 2: The "Code Chef" (Writing the Rule)

Now that the AI has a bunch of examples (Jan 1 $\to$ Jan 2, June 15 $\to$ June 16, etc.), it tries to write the actual "magic formula" (the code) that turns any Date A into Date B.

However, AI chefs sometimes add extra ingredients they don't need, or even use ingredients that don't exist in your kitchen (like trying to use a "flying spoon").

Refinement: MR-Adopt acts like a strict head chef. It looks at the code the AI wrote and cuts out all the useless or broken parts. It keeps only the steps that actually turn the date forward.
Testing: Finally, it runs the new rule against many different dates (like a test kitchen). If the rule works for every date, it keeps it. If it only works for January dates, it throws it away and tries again.

The Result: A Factory That Runs Itself

Once MR-Adopt figures out the rule (e.g., "Add one day"), it can apply that rule to any date you throw at it.

Before: You could only test the two dates the original baker wrote down.
After: You can test thousands of dates automatically.

The paper shows that by using MR-Adopt:

More Bugs Found: They found 18.9% more bugs (mutation score) than before.
Better Coverage: They tested 10.6% more of the code (line coverage).

Why This Matters

Think of it like upgrading from a manual car to an autonomous car.

Old Way: You have to manually steer the car (write test inputs) for every single turn. If you forget a turn, you miss a spot.
MR-Adopt Way: You teach the car the rules of the road (the transformation). Now, the car can drive itself through any new street, checking for potholes (bugs) everywhere, without you needing to write down every single turn in advance.

In short, MR-Adopt takes old, rigid test notes and turns them into flexible, automatic instructions, making software testing much faster, smarter, and more thorough.

1. Problem Statement

Metamorphic Testing (MT) is a technique used to address the "oracle problem" (the difficulty of determining the correct output for a given input) by verifying relationships between inputs and outputs rather than specific output values. A Metamorphic Relation (MR) defines how a follow-up input ( $x_f$ ) relates to a source input ( $x_s$ ) and how their respective outputs ( $y_s, y_f$ ) should relate.

While recent studies show that developers often encode MRs within existing test cases, a critical limitation exists: over 70% of these "MR-encoded test cases" (MTCs) hard-code specific source and follow-up input values (e.g., dateA = "2024-01-01", dateB = "2024-01-02") without an explicit input transformation function to generate $x_f$ from $x_s$ .

Consequence: Without an explicit transformation function, these valuable encoded MRs cannot be reused with automatically generated source inputs. This prevents the automation of MT and limits the ability to enhance test adequacy (coverage and fault detection) across diverse inputs.
Goal: The paper aims to automatically deduce an explicit input transformation function from hard-coded input pairs within an MTC, enabling the generalization of these MRs to new, automatically generated source inputs.

2. Methodology: MR-Adopt

The authors propose MR-Adopt (Automatic Deduction Of inPut Transformation), a two-phase pipeline leveraging Large Language Models (LLMs) to infer and generate input transformation functions. The system operates on the premise that while LLMs struggle with "Programming by Example" (PBE) given only a single example, they can generalize better with multiple examples and refined code.

Phase 1: Input Pair Preparation

The goal is to generate additional source-follow-up input pairs to better illustrate the underlying input relation, mitigating overfitting to the single hard-coded example.

Generation: The LLM is prompted to generate new source inputs and corresponding follow-up inputs based on the context of the MTC (including the hard-coded pair, the method under test, and the output assertion). This is done in two steps: first generating source inputs, then follow-up inputs.
Refinement (Data-Flow Analysis): LLMs often generate irrelevant or erroneous code (e.g., incorrect assertions or type mismatches). MR-Adopt performs data-flow analysis on the generated code to build a dependency graph. It isolates only the statements required to construct the source and follow-up inputs, removing irrelevant code segments that could cause compilation errors.
Validation: The refined input pairs are executed against the System Under Test (SUT). The outputs are checked against the developer-written output relation (assertions) in the original MTC. Only pairs that satisfy the output relation are retained as valid examples.

Phase 2: Transformation Generation

Using the original hard-coded pair and the newly generated valid pairs, the LLM is tasked with synthesizing a generalizable transformation function.

Generation: The LLM generates candidate transformation functions (code that takes a source input and returns a follow-up input).
Refinement: Similar to Phase 1, data-flow analysis is used to remove irrelevant code and ensure the function only contains logic necessary for the transformation. Dependencies (imports) are automatically resolved.
Assessment: The candidate transformations are tested against a set of new, unseen source inputs. The system selects the transformation that successfully generates valid follow-up inputs for the highest percentage of these new inputs.

3. Key Contributions

First Approach for Deduction: To the best of the authors' knowledge, this is the first work to automatically generate input transformation functions for MRs encoded in existing test cases.
MR-Adopt Framework: A novel LLM-based approach that combines:
- Analogical Reasoning: Generating multiple example input pairs to guide the LLM and prevent overfitting.
- Code Refinement: Using data-flow analysis to strip irrelevant/erroneous code from LLM outputs.
- Validation Strategy: Using developer-written output assertions as oracles to filter valid pairs and select the most generalizable transformation.
Dataset: The creation and release of a dataset containing 100 encoded MRs (dated after April 2023 to avoid data leakage) with ground-truth transformations for evaluation.

4. Experimental Results

The authors evaluated MR-Adopt on 100 MTCs using various LLMs (GPT-3.5, Llama3, Deepseek).

Effectiveness of Generation (RQ1):
- MR-Adopt generated compilable transformations for 95% of the 100 MRs.
- 72% of the generated transformations were 100% generalizable (applicable to all prepared source inputs).
- Compared to directly prompting GPT-3.5, MR-Adopt improved the generation of 100% generalizable transformations by 33.33%.
Effectiveness in Constructing Follow-up Inputs (RQ2):
- Transformations generated by MR-Adopt produced valid follow-up inputs for 91.21% of source inputs.
- This represents a 75.99% improvement over using LLMs directly as black-box transformations (which only achieved ~43-53% validity).
Ablation Study (RQ3):
- All three components (additional input pairs, refinement, and assessment) contributed to performance.
- The assessment step (selecting the best transformation) had the highest impact, followed closely by the additional input pairs (mitigating overfitting).
Test Adequacy Enhancement (RQ4):
- Integrating MR-Adopt's transformations with new source inputs significantly improved test quality.
- Line Coverage increased by 10.62%.
- Mutation Score increased by 18.91% compared to the original developer-written test cases.

5. Significance

Unlocking Legacy Test Cases: MR-Adopt transforms static, hard-coded test cases into dynamic, reusable testing assets. It allows developers to leverage the domain knowledge embedded in existing tests without manually rewriting them.
Solving the Oracle Problem: By automating the generation of follow-up inputs based on verified MRs, the technique enables robust testing of "non-testable" programs (e.g., encryption, time-dependent systems) where expected outputs are hard to define.
Practical Utility: The significant increase in line coverage and mutation score demonstrates that this approach effectively uncovers faults that traditional random testing or single-pair testing misses.
Generalizability: The methodology of using LLMs for code generation, refined by static analysis and validated by execution, offers a blueprint for other code synthesis tasks where LLM hallucinations are a concern.

In summary, MR-Adoot bridges the gap between the theoretical power of Metamorphic Testing and the practical limitations of hard-coded test cases, significantly enhancing software testing automation through the intelligent deduction of input transformations.