Large Language Models for Multilingual Code Intelligence: A Survey

1. Problem Statement

The paper addresses a critical gap in current AI-assisted software engineering: the "monolingual bias" of Large Language Models (LLMs). While state-of-the-art models (e.g., GPT-5, DeepSeek, Qwen3) exhibit near-human proficiency in high-resource languages like Python and Java, their performance degrades significantly in system-level, functional, or low-resource languages (e.g., Rust, OCaml, C/C++, Lua).

This disparity creates a bottleneck because real-world software systems are inherently polyglot, often combining multiple languages within a single stack (e.g., Java for services, C++ for kernels, Rust for memory safety). Current research fails to provide robust multilingual code intelligence, specifically struggling with:

• Cross-paradigm generalization: Adapting logic from one language paradigm to another (e.g., object-oriented to functional).
• Semantic preservation: Ensuring that translated or generated code behaves identically across different languages with distinct memory models and type systems.
• Evaluation asymmetry: The lack of standardized benchmarks that account for the unique compilation, linking, and runtime constraints of diverse languages.

2. Core Tasks Formalized

The survey organizes the field around two complementary pillars:

1. Multilingual Code Generation ($F_{gen}: NL \to \{PL_1, \dots, PL_n\}$):

   • Goal: Synthesizing functional source code from natural language (NL) requirements in one or more target languages.
   • Challenge: Mapping abstract human intent to specific syntactic and idiomatic constraints of diverse languages simultaneously.
   • Use Case: Rapid prototyping where a single algorithm must be implemented in Python (for testing) and Rust/C++ (for production).

2. Multilingual Code Translation ($F_{trans}: PL_{source} \to PL_{target}$):

   • Goal: Converting existing source code from a source language to a target language while preserving semantics.
   • Challenge: Achieving semantic equivalence despite divergent programming paradigms (e.g., manual memory management in C vs. ownership in Rust).
   • Use Case: Modernizing legacy code (C/C++ to Rust for security) or cross-platform porting (Desktop C++ to Android Java/Kotlin).

3. Methodological Landscape

The authors categorize existing approaches into four paradigms, arguing that they are complementary rather than mutually exclusive:

• Prompt Engineering:

  • Mechanism: Uses frozen LLMs with zero-shot or few-shot prompting, in-context learning, and structural scaffolds.
  • Techniques: Includes "NL in the Middle" (using natural language as a pivot), "Pseudocode" generation to decouple logic from syntax, and transitive intermediate translations.
  • Limitation: Bounded by the model's pretraining distribution; struggles with deep semantic shifts in low-resource languages.

• Model Pre-training and Fine-tuning:

  • Mechanism: Internalizes multilingual competence by modifying model parameters via massive data collection or synthetic generation.
  • Techniques: Utilizing compiler Intermediate Representations (IR) as language-agnostic bridges (e.g., IRCoder) and Open-Source Synthetic Instruction (OSS-Instruct).
  • Limitation: Data scarcity and quality remain bottlenecks for low-resource languages; high computational cost.

• Multi-Agent Collaborative Frameworks:

  • Mechanism: Decomposes complex tasks into iterative cycles involving specialized agents (e.g., planner, coder, validator).
  • Techniques: Simulates software engineering workflows (e.g., "validate and repair" loops) to handle cross-file dependencies and build configurations (e.g., RepoTransAgent, MatchFixAgent).
  • Advantage: Provides robustness for repository-level tasks and long-tail languages by leveraging feedback loops.

• Retrieval-Augmented Generation (RAG):

  • Mechanism: Injects external, language-specific knowledge (libraries, APIs, dependencies) into the inference context.
  • Techniques: Retrieving class definitions, dependencies, or parallel code pairs from corpora to ground the model.
  • Advantage: Mitigates hallucination in specialized or emerging languages by providing project-aware context.

4. Evaluation Practices and Benchmarks

The paper highlights that multilingual evaluation is fundamentally harder than monolingual evaluation due to semantic relativity (behavior may differ slightly across languages) and environmental asymmetry (toolchains vary).

• Metrics:
  • Execution-based: Pass@k (probability of passing unit tests) and Computational Accuracy (CA) (exact input-output match). These are the gold standard for functional correctness.
  • Semantic/Textual: CodeBLEU and LLM-as-a-Judge. The latter is gaining traction to evaluate readability, maintainability, and adherence to complex instructions where static metrics fail.
• Key Benchmarks:
  • Generation: mHumanEval (200+ languages), HumanEval-XL, PSEUDOEVAL.
  • Translation: RepoTransBench (repository-level), XLCoST, CodeTransOcean (45 languages), G-TransEval.
  • Trend: Benchmarks are evolving from isolated snippets to repository-level complexity, requiring reasoning over cross-file dependencies and build systems.

5. Key Contributions

1. Taxonomy of Methods: A structured classification of multilingual code intelligence approaches (Prompting, Fine-tuning, Multi-Agent, RAG) with a comparative analysis of their strengths and failure modes.
2. Task Formalization: Clear mathematical definitions distinguishing between Generation (intent-to-code) and Translation (code-to-code), emphasizing the unique challenges of semantic preservation.
3. Comprehensive Benchmark Review: A critical survey of 15+ datasets, analyzing their evolution from lexical matching to execution-based verification and repository-level complexity.
4. Identification of Gaps: Highlights the "long-tail" problem where academic benchmarks fail to capture the complexity of domain-specific libraries and legacy systems found in industry.

6. Results and Findings

• Performance Disparity: Significant gaps exist between high-resource (Python) and low-resource (Rust, OCaml) languages, even in top-tier models.
• Directional Asymmetry: Translation is often harder when moving into stricter type systems (e.g., C to Rust) than into flexible ones (e.g., Rust to Python).
• Hybrid Necessity: No single paradigm solves the problem. The most effective approaches combine parameter-level competence (fine-tuning) with structured reasoning (multi-agent) and external grounding (RAG).
• Implementation Bottleneck: For low-resource languages, the primary barrier is no longer just semantic understanding, but the mastery of precise, low-level implementation details and idioms.

7. Significance

This survey provides a foundational roadmap for the next generation of trustworthy polyglot coding systems. It shifts the focus from merely generating code in popular languages to enabling robust cross-language generalization, which is essential for:

• Software Modernization: Safely migrating legacy systems to memory-safe languages.
• Cross-Platform Development: Enabling "write once, generate everywhere" workflows.
• Industrial Reliability: Reducing the risk of semantic errors in critical infrastructure that relies on mixed-language stacks.

The authors conclude that achieving true multilingual code intelligence requires a holistic approach that addresses data imbalance, improves evaluation rigor, and leverages collaborative AI architectures to handle the complexity of real-world software engineering.