MR-Coupler: Automated Metamorphic Test Generation via Functional Coupling Analysis

1. Problem Statement

Metamorphic Testing (MT) is a technique designed to solve the "test oracle problem" (where expected outputs are difficult to define) by verifying relations between multiple inputs and their corresponding outputs. However, the widespread adoption of MT is hindered by the difficulty of constructing effective Metamorphic Relations (MRs).

• Current Limitations: Existing approaches often rely on manual effort, domain-specific knowledge, regression testing scenarios, or high-quality specifications that are rarely available.
• The Gap: There is a lack of fully automated, domain-agnostic methods to generate concrete Metamorphic Test Cases (MTCs) directly from source code without requiring pre-existing test cases or external documentation.

2. Methodology: MR-Coupler

The authors propose MR-Coupler, an automated framework that leverages functional coupling between methods within a class to construct MRs. The core hypothesis is that methods with functional relationships (e.g., inverse, equivalent, or producer-consumer) naturally form the basis for valid MRs.

The approach operates in three distinct phases:

Phase 1: Coupled Methods Identification

Instead of blindly enumerating all possible method pairs (which is computationally expensive and noisy), MR-Coupler identifies functionally coupled method pairs using three specific features derived from code analysis:

1. Signature Commonality (Intent): Analyzes method names, parameter types, and return types.
   • Example: encrypt and decrypt share complementary types, suggesting an inverse relationship.
   • Example: Overloaded methods suggest equivalence relations.
2. Function Call (Behavior): Analyzes direct call dependencies (Method A calls Method B) or shared API invocations.
   • Example: Two methods calling the same underlying library functions suggest functional similarity.
3. State Interaction: Analyzes read/write interactions with shared class fields.
   • Example: One method writes to a field that another reads, suggesting a data dependency suitable for MR construction.

Phase 2: Candidate MTC Generation

Once coupled pairs are identified, the system employs Large Language Models (LLMs) to generate concrete test cases.

• Contextual Guidance: To prevent LLM hallucinations (e.g., non-existent APIs), the prompt includes:
  • Code of the paired methods.
  • Identified coupling features.
  • Real-world invocation examples retrieved from the project's existing test suite.
  • A strict MTC template.
• Refinement: Generated code that fails to compile or execute is automatically refined. The system uses static analysis (JavaParser) to fix missing imports and retries the LLM with error messages until the code is executable.

Phase 3: MTC Amplification and Validation

To ensure the generated MTCs are valid and not false positives, the system performs:

• Input Augmentation: The LLM is instructed to generate additional test inputs (e.g., boundary values, special characters) to exercise the MR across diverse scenarios.
• Mutation-Based Validation:
  • The system creates mutants of the target program by injecting artificial faults.
  • It executes the amplified MTCs on both the original (correct) version and the mutant (faulty) version.
  • Filtering Logic: A valid MTC should pass on the original version (high pass rate) and fail on the mutant version (low pass rate). If the pass rates are inverted (failing on original, passing on mutant) or identical, the MTC is discarded as a false alarm or invalid.

3. Key Contributions

1. Novel Paradigm: First approach to construct MRs by leveraging functional coupling between methods, relying solely on source code rather than external specifications or existing test cases.
2. MR-Coupler Tool: A fully automated pipeline that integrates static analysis, LLM reasoning, and mutation testing to generate, refine, and validate MTCs.
3. Validation Mechanism: A novel filtering strategy using mutation analysis to significantly reduce false alarms without requiring ground truth oracles.
4. Comprehensive Evaluation: Extensive experiments on 100 human-written MTC tasks and 50 real-world bugs, alongside ablation studies and cost analysis.
5. Open Source: Release of the tool, datasets (including human-written MTCs and real-world bugs), and experimental data.

4. Experimental Results

The evaluation was conducted on 100 target methods and 50 real-world bugs using various state-of-the-art LLMs (GPT-4o-mini, Qwen3-coder-Flash, DeepSeek-V3.1, DeepSeek-V3.1-Think).

• Validity: MR-Coupler successfully generated valid MTCs for over 90% of tasks (specifically 98/100 with the best model).
• Improvement over Baselines:
  • 64.90% improvement in the rate of valid MTC generation compared to baselines (direct LLM prompting without coupling analysis).
  • 36.56% reduction in false alarms.
• Bug Detection: The generated MTCs successfully detected 44% (22 out of 50) of the real-world bugs.
• Similarity to Human Practice: MR-Coupler achieved >90% consistency with human-written MTCs regarding the "MR-skeleton" (the structure of input transformation, method pairing, and output assertion), indicating it mimics developer intuition effectively.
• Ablation Study:
  • Removing Coupling Analysis caused a 32.65% drop in valid MTCs and a 56.62% drop in bug detection.
  • Removing Input Amplification reduced bug detection by 40.91%.
  • Removing Validation increased false alarms by 71.56%.
• Cost: The average cost per target method ranges from $0.09 to $0.40 (depending on the LLM), consuming approximately 500k–730k tokens.

5. Significance and Impact

• Lowering the Barrier: MR-Coupler removes the need for domain-specific knowledge or pre-existing test cases, making Metamorphic Testing accessible for a wider range of software systems.
• Practical Applicability: By achieving high consistency with human-written tests and effectively detecting real bugs, the tool demonstrates potential for assisting developers in test construction or even partially replacing manual effort.
• Complementarity: Unlike previous tools (e.g., MR-Scout) that require existing test cases to synthesize MRs, MR-Coupler works on programs with no existing MT infrastructure, filling a critical gap in automated testing.
• Future Direction: The work highlights the potential of combining static code analysis (for coupling) with LLM reasoning (for relation formulation) to solve complex software engineering problems.

In conclusion, MR-Coupler represents a significant step forward in automating metamorphic testing, transforming a traditionally knowledge-intensive task into a code-driven, automated process that effectively uncovers real-world software defects.