CrossLangFuzzer: Differential Testing of Cross-Language JVM Compilers

Technical Summary: CrossLangFuzzer

Problem Statement
Modern Java Virtual Machine (JVM) ecosystems increasingly rely on the interoperability of multiple programming languages, such as Java, Kotlin, Groovy, and Scala, within single applications. While these languages coexist, they possess subtle semantic differences regarding type systems, nullability models, generics, variance rules, and method dispatch. Existing compiler testing techniques predominantly focus on isolated, single-language compilation pipelines. Consequently, they fail to systematically exercise the diverse semantic interactions that occur at language boundaries. This gap leaves a significant class of cross-language compilation bugs—where compilers misinterpret dependencies or semantic discrepancies between languages—largely unexplored and undetected.

Methodology
To address this gap, the authors present CrossLangFuzzer, the first differential testing framework designed specifically for cross-language JVM compilation. The system operates through a four-stage automated pipeline:

1. Unified Intermediate Representation (IR) Generation:
   Instead of generating source code directly, CrossLangFuzzer synthesizes test programs using a custom, unified IR inspired by the Kotlin compiler's backend. This IR decouples test generation from language-specific syntax, modeling a hierarchical declaration tree that captures multi-language semantics (e.g., a Scala class extending a parameterized Java interface). The IR supports five type forms: simple types, parameterized types, nullable types, platform types (handling ambiguous nullability like String!), and type parameters with upper bounds. The generator ensures that initial programs are structurally valid by enforcing JVM inheritance rules and semantic constraints.

2. Mutation Strategy:
   To increase test diversity and bug-finding capability, the framework applies seven specific mutation operators to the valid IR. These operators target four key areas of cross-language divergence:

   • Generic Subtyping: Altering type arguments and bounds to test variance and substitution logic.
   • Nullability: Toggling nullability modifiers on parameters and type bounds to stress-test cross-language nullability coercion.
   • Override Resolution: Stripping method bodies to test linkage and resolution rules across languages.
   • Language Placement: Reassigning the target language of specific classes within the same structure to amplify semantic inconsistencies.
     Unlike the generator, the mutator does not enforce semantic well-typedness, allowing the system to explore ill-typed programs where compiler divergence (accept vs. reject) may indicate bugs.

3. Differential Testing and Execution:
   The mutated IR is translated into concrete source code (Kotlin, Java, Groovy, or Scala) using specific printers. The framework then executes two testing modes:

   • Normal Testing: Compiling code with a single compiler version to detect crashes or internal errors.
   • Differential Testing: Compiling the same code across different compiler versions or different language compilers. Behavioral mismatches (e.g., one compiler accepts code while another rejects it, or they produce different bytecode) are flagged as potential bugs.

4. Automated Reduction:
   When a bug is detected, CrossLangFuzzer serializes the triggering program back into the unified IR. It employs an optimized Delta Debugging Minimization (DDMin) algorithm with an active validator to reduce the test case. The reducer iteratively removes elements from the IR, ensuring structural validity is maintained, until the minimal set of code required to reproduce the bug is identified. This minimized IR is then converted back to source code for reporting.

Key Contributions

• First Differential Framework for Cross-Language JVM: CrossLangFuzzer is the first tool to systematically test cross-language compilation by leveraging a unified IR to synthesize and mutate multi-language programs.
• Unified IR Architecture: The introduction of a language-agnostic IR that models complex JVM interoperability features (generics, nullability, variance) allows for flexible code generation across multiple target languages without relying on a single compiler's frontend.
• Targeted Mutation Operators: The definition of seven mutation operators specifically designed to expose semantic discrepancies in cross-language contexts.
• Automated Bug Reduction: The integration of an IR-based minimizer that reduces complex cross-language test cases to minimal, reproducible examples.

Results
Evaluated on the latest versions of five major JVM compilers (Kotlin, Groovy, Scala 3, Scala 2, and Java), CrossLangFuzzer uncovered 32 confirmed bugs:

• Kotlin: 15 bugs (1 fixed, 14 confirmed).
• Groovy: 4 bugs (all 4 fixed).
• Scala 3: 7 bugs (confirmed).
• Scala 2: 2 bugs (confirmed).
• Java: 4 bugs (confirmed).

The tool successfully identified defects in actively maintained compiler versions, demonstrating that cross-language inconsistencies remain a significant source of errors even in mature ecosystems.

Significance
The paper claims that CrossLangFuzzer effectively addresses the lack of systematic testing for cross-language compilation boundaries. By moving beyond single-language isolation, the framework exposes a class of bugs related to semantic reconciliation that existing tools miss. The immediate actionability of the findings is highlighted by the fact that all reported issues were verified by compiler development teams, with several already fixed. The authors position the tool as a critical step toward improving the robustness and interoperability of the JVM ecosystem. Additionally, the authors note that the use of a serialized IR provides a natural interface for future integration with Large Language Models (LLMs) to assist in fuzzing, though this is identified as future work rather than a current capability.