Technical Summary: Neural Code Translation of Legacy Code from APL to C#

Problem Statement

The migration of legacy systems from specialized, array-oriented languages to modern, statically typed object-oriented languages presents significant challenges for maintainability and integration. This paper addresses the specific problem of translating APL (A Programming Language) into C#. The task is inherently difficult due to three primary factors:

1. Syntactic Disparity: APL utilizes a sparse syntax with unique glyphs and operates on multidimensional arrays without explicit loops, whereas C# relies on verbose, explicit procedural logic and distinct type signatures.
2. Data Scarcity: There is a lack of large-scale, parallel corpora containing functionally equivalent APL and C# code pairs, which limits the training of standard neural translation models.
3. Semantic Ambiguity: APL's dynamic typing and rank-polymorphic functions (where a single function operates on scalars, vectors, and matrices) create ambiguity when mapping to C#'s strict type system, often requiring complex method overloading or runtime type checking to preserve functionality.

Methodology

The study treats code translation as a sequence-to-sequence generation problem using Large Language Models (LLMs). To overcome data limitations and structural ambiguities, the authors propose and evaluate a framework comparing four distinct translation strategies against a baseline:

1. Direct Translation (Baseline): Fine-tuning an LLM to translate APL code directly to C# without intermediate guidance.
2. Natural Language (NL) Mediation: A two-stage approach where an LLM first generates a high-level natural language description of the APL code, which is then combined with the source code to generate the C# translation.
3. Retrieval-Augmented Generation (RAG): The system retrieves relevant APL documentation and idioms from external libraries (e.g., APL2 Idiom Library) to provide context to the model during generation.
4. Iterative Refinement: An automated loop where generated C# code is compiled and executed against test cases. Compilation errors, runtime exceptions, and functional mismatches are fed back to the model as context for subsequent refinement iterations.

Dataset Construction:
Due to the scarcity of existing data, the authors constructed multiple datasets:

• Dataset A: 800 manually curated fundamental APL-C# pairs.
• Dataset B: 143 complex utility functions derived from production environments.
• Dataset C: 320 cross-language implementations from Rosetta Code.
• Dataset I: 45 APL idioms with descriptions.
  A total of 1,066 samples were used for fine-tuning, with Dataset B serving as the primary testbed for production-oriented generalization.

Technical Implementation:

• Tokenization: The study analyzed tokenizer efficiency for APL glyphs, identifying Qwen3-32B and Gemma-4-31b-it as the most efficient models for handling APL characters without round-trip failures.
• Fine-Tuning: Parameter-Efficient Fine-Tuning (LoRA) with 8-bit quantization was employed to fit models within available H100 GPU memory constraints.
• Evaluation Pipeline: An automated pipeline verified translation quality by checking for successful compilation and functional equivalence against input-output test cases.

Key Contributions

• First Systematic Investigation: This work presents the first systematic study and specialized dataset for neural code translation from APL to a modern language (C#).
• Guided Translation Framework: The paper introduces and empirically validates a framework that combines method signatures (extracted via an F# parser), natural language descriptions, and iterative feedback loops to bridge the paradigm gap between APL and C#.
• Automated Evaluation: The development of a rigorous evaluation pipeline that measures both syntactic correctness (compilation) and functional correctness (execution) for generated code.

Experimental Results

The experiments were conducted on a test set of 49 production-grade APL functions (from Dataset B). Key findings include:

• Impact of Guidance: The inclusion of C# method signatures significantly improved compilation rates across all models. For instance, adding signatures to the Gemma-4-31b-it model increased the full pass rate from 22.45% to 46.94%.
• Effectiveness of Iteration: The Iterative Translation strategy yielded the highest performance. The combination of NL descriptions and iterative feedback achieved a peak Full Pass Rate of 79.59% using Claude Opus 4, followed closely by 77.55% using Gemma-4-31b-it.
• Model Performance: Fine-tuned open-weight models (Gemma-4-31b-it, Qwen3-32B) with guidance strategies achieved performance competitive with closed-weight models (GPT-5, Claude Opus 4).
• Error Analysis: The primary cause of failure shifted from compilation/runtime errors in early iterations to functional errors in later iterations. This indicates that while models can learn to generate syntactically valid C#, preserving the specific mathematical intent of APL idioms remains the most challenging aspect.
• Data Scaling: Performance metrics showed a consistent upward trend with increased training data size, suggesting that current results are limited by dataset scale rather than model architecture.

Significance and Claims

The paper claims that neural code translation can successfully bridge the gap between legacy APL systems and modern C# architectures, provided that appropriate guidance strategies are employed. The authors assert that:

1. Feasibility: Automated translation is viable for a wide range of programs, moving beyond simple syntax conversion to functional equivalence.
2. Cost-Effectiveness: By utilizing fine-tuned open-weight models combined with iterative refinement, organizations can achieve results comparable to expensive closed-source models, offering a scalable solution for legacy modernization.
3. Practical Applicability: The methodology reduces the manual effort required to debug and refine migrated code, addressing the diminishing pool of APL experts.

The authors maintain a modest stance, noting that while the approach is effective, the majority of remaining failures stem from a misunderstanding of the original APL intent rather than low-level code generation errors. They conclude that future work should focus on expanding training corpora and transitioning from single-program translation to multi-agent systems capable of handling project-level dependencies.