Technical Summary: BatCoder

Problem Statement

Training Large Language Models (LLMs) for code-related tasks, specifically the bidirectional transformation between natural language documentation and executable code, typically relies on high-quality, curated code-documentation pairs. Such paired data is costly to create and often scarce, particularly for niche or low-resource programming languages. While raw source code is abundant in public repositories, the lack of explicit supervision constrains the scalability and generalization of models on these tasks. Existing data augmentation strategies often require stronger external "teacher" models to synthesize documentation or pseudo-labels, preventing the target model from leveraging the same mechanism for self-improvement. Furthermore, standard optimization methods often treat generated documentation as fixed supervision rather than explicitly optimizing it against the training objective.

Methodology

The authors propose BatCoder, a self-supervised reinforcement learning framework that jointly optimizes code generation and documentation production using only unlabeled code snippets. The core innovation is a back-translation strategy where the model learns to generate documentation from code and subsequently reconstruct the original code from that documentation.

Framework Overview

The training process consists of two sequential stages:

1. Stage 1 (Code-to-Documentation): Given an unlabeled code snippet $c$, the model generates a natural language document $d$.
2. Stage 2 (Documentation-to-Code): The generated document $d$ is used to reconstruct a code snippet $c'$.

The framework treats the composition of these stages ($g_\theta \circ f_\theta$) as a reconstruction objective. The semantic and structural similarity between the original code $c$ and the reconstructed code $c'$ serves as an implicit reward signal. This allows the model to optimize both the documentation generation and the code reconstruction without external paired data.

Key Technical Components

• Asymmetric Sampling Strategy: During training, the model samples $K$ documentation candidates for a single code snippet (Stage 1) but generates only a single reconstructed code sample for each valid document (Stage 2). This balances trajectory diversity with computational efficiency.
• Reward Design:
  • Code Similarity Reward: The primary signal is derived from a code-level similarity metric $S(c, c')$. The authors instantiate this using CSSG (Code Similarity based on an improved Program Dependence Graph), which scores semantic and structural similarity on a scale of [0, 1].
  • Documentation Validity Reward: An auxiliary reward ($R_{doc}$) is applied in Stage 1 to ensure the generated documentation adheres to a specific format and does not contain redundant content. This stabilizes the training process.
  • Combined Reward: The reward for Stage 1 is the product of the reconstruction similarity and the validity reward. If a document fails validity checks, the reward is set to zero, and no reconstruction is attempted.
• Optimization Algorithm: The framework employs Reinforce++, a policy gradient algorithm. It utilizes a replay buffer to store trajectories and computes token-level advantage estimates based on normalized rewards. The loss function jointly optimizes both generation directions with a KL regularization term to constrain policy deviation from a reference model.

Key Contributions

1. Self-Supervised Back-Translation Framework: BatCoder enables bidirectional learning between code and documentation without relying on externally curated paired data, effectively alleviating the scarcity of supervision.
2. Scalability: The authors empirically demonstrate that BatCoder exhibits favorable scaling behavior with respect to both model capacity (tested on 3B and 7B parameters) and training data size, showing that reconstruction-based self-supervision provides increasingly effective signals as scale increases.
3. Performance in Low-Resource Languages: The framework consistently improves performance on both code generation and documentation tasks, with particularly strong gains observed in low-resource programming languages where paired data is scarce.

Experimental Results

The framework was evaluated using Qwen2.5-3B and Qwen2.5-7B as base models, trained on code data from the CodeXGLUE benchmark (Python, Ruby, Go).

• Python Benchmarks (HumanEval/MBPP):
  • The 7B BatCoder model achieved 83.5% pass@1 on HumanEval and 81.0% on MBPP.
  • This outperforms the base Qwen2.5-7B-Instruct model and surpasses larger open-source baselines, including the 33B DeepSeek-Coder-Instruct on HumanEval.
  • Similar improvements were observed at the 3B scale.
• Low-Resource Languages (MultiPL-E):
  • Ruby: The base 3B model achieved 0.0% pass@1. BatCoder raised this to 10.6%. The 7B model improved from 3.1% to 13.0%.
  • Go: Consistent improvements were seen, with the 7B model increasing pass@1 from 34.4% to 39.0%.
• Ablation Studies:
  • Removing the Stage 1 optimization (training only Stage 2) resulted in marginal gains, highlighting the necessity of jointly optimizing documentation generation.
  • Comparison with Supervised Fine-Tuning (SFT) on synthetic data showed that BatCoder's reinforcement learning approach, which explicitly evaluates documentation quality via reconstruction, yields superior results compared to standard SFT on pseudo-pairs.

Significance and Claims

The paper posits that BatCoder offers a robust solution to the data scarcity problem in code-related LLM training. By leveraging the inherent consistency between code and its documentation, the framework allows models to learn effectively from raw, unlabeled code. The authors claim that this approach not only improves performance on standard benchmarks but is particularly transformative for low-resource programming languages, where traditional supervised methods struggle due to a lack of high-quality paired data. The work demonstrates that back-translation similarity can serve as a reliable, self-contained learning signal, enabling models to improve their bidirectional capabilities without external teacher models or curated datasets.