🤖 machine learning

Selective Left-Shift: Turning Test-Time Compute and Difficulty-based Curation into Training Data for Low-Resource Code Generation

This paper proposes a three-phase pipeline that decouples syntax acquisition from algorithmic reasoning by shifting test-time compute into offline data synthesis and combining supervised fine-tuning with execution-grounded reinforcement learning, thereby significantly improving code generation performance for low-resource programming languages in small language models while reducing data requirements and costs.

Original authors: Didula Samaraweera, Anjana Supun, Srinath Perera

Published 2026-07-10

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Didula Samaraweera, Anjana Supun, Srinath Perera

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you're trying to teach a brilliant but inexperienced apprentice how to write code in a very rare, obscure language—like Julia (used by scientists) or Ballerina (used for cloud services). The problem? The apprentice has never seen these languages before. They know Python and Java inside out, but when asked to write in Julia, they start mixing up the grammar, using the wrong punctuation, and getting lost in the syntax.

This is the "trilemma" the authors of this paper faced:

Not enough data: There are very few examples of good code in these rare languages to teach the model.
Too expensive to think: You could make the model "think harder" and try again and again while you wait (like a human debugging), but that takes too much time and money for real-world use.
Reinforcement Learning fails: Trying to teach the model by rewarding it for good code and punishing bad code doesn't work well if the model is making so many basic grammar mistakes that it never gets a "good job" signal to learn from.

The Big Idea: "Left-Shift" the Homework

The authors propose a clever three-step pipeline they call "Selective Left-Shift." Think of it like this: instead of making the apprentice struggle through every single homework assignment while you are watching (which is slow and expensive), you take all that struggle, do it once in a quiet room offline, and turn the results into a perfect textbook.

Here is how their three-phase pipeline works, using the analogy of training a coding wizard:

Phase 1: The Offline "Try-Fail-Try" Factory

Instead of waiting for the model to fail in real-time, the authors set up an automated factory. They give the model a problem and let it try to solve it.

The Magic: If the code fails, the factory doesn't just throw it away. It feeds the error message (like "You used a comma instead of a semicolon" or "This test case failed because the output was 5, not 6") back to the model.
The Loop: The model tries again, fixes the error, and tries again. It keeps looping until the code passes all the tests.
The Result: Once the code works perfectly, it gets saved into a special "Verified Dataset." The authors call this left-shifting the compute. They do all the expensive "thinking and fixing" once to create high-quality training data, rather than doing it every time a user asks a question later.

Phase 2: The Grammar Bootcamp (SFT)

Now that they have a textbook full of perfectly verified code, they teach the model (specifically a Qwen3-8B model) using Supervised Fine-Tuning (SFT).

Why this matters: Before this, the model was making so many syntax errors (like writing True instead of true in Julia) that it couldn't even get to the logic part. This phase is like a strict grammar bootcamp. It forces the model to learn the specific "dialect" of Julia or Ballerina so it stops making silly grammar mistakes.
The Proof: The paper shows that after this step, syntax errors drop from 45.9% to 0.6%. The model can now write code that actually compiles.

Phase 3: The "Deliberate Practice" Gym (RLVR)

Now the model knows the grammar. It's time to teach it the logic—how to actually solve the hard problems. They use Reinforcement Learning (RL), but with a twist.

The Twist (Difficulty Curation): Most RL methods throw random problems at the model. The authors argue this is inefficient. If a problem is too easy, the model solves it instantly and learns nothing. If it's too hard, the model fails everything and learns nothing.
The Strategy: They curate a dataset of problems that are just right—right at the edge of what the model can do (measured by ELO ratings, a score used in chess and competitive programming). This is like "deliberate practice" for athletes: you only train on the moves you are almost good at, so you improve fastest.
The Safety Net: They also use a trick called Zero-Advantage Masking. If a group of attempts all fail (or all succeed), they ignore that group. They only learn from groups where some attempts worked and others didn't, ensuring the model gets a clear signal on what to improve.

The Results: A Massive Leap Forward

The results are impressive, especially considering they used 1/3 of the data and 1/6 of the cost compared to previous state-of-the-art methods.

For Julia (a "moderate" rare language):
- The base model scored 44.0% on standard tests (MultiPL-E).
- After their full pipeline, it jumped to 68.6%. That's a +24.6 point improvement.
- On harder, real-world tests (Ag-LCB), it went from 9% to 39.2%.
- They beat the previous best results by 7.6 points on standard tests and 14.2 points on hard tests.
For Ballerina (an "extreme" rare language):
- The base model knew almost nothing about Ballerina (scoring 4.4%).
- After the pipeline, it scored 49.7%.
- This proves the method works even for languages the model has never seen before, as long as you have a compiler and some test cases.

What They Explicitly Rule Out

The paper is very clear about what doesn't work well on its own:

Just Reinforcement Learning (RL) from scratch: They argue that if you try to teach a model RL without first teaching it the grammar (SFT), it fails. The model gets stuck making syntax errors and never gets the "reward" signal to learn logic.
Random Difficulty: They tested using random problems for the RL phase and found it much less effective (52.2% vs 68.6%). They suggest that picking the right difficulty level is crucial for the learning to stick.
Online "Think Harder" Scaling: They argue that doing the "try-fail-fix" loop during live usage (inference-time scaling) is too expensive and slow. Their "offline" approach is the better investment.

How Sure Are They?

The authors are quite confident in their findings because they measured them directly.

They proved that the pipeline works by testing it on two different languages (Julia and Ballerina) and two different benchmarks.
They measured the cost savings, showing their method cost $54.02 compared to $320.3 for a similar previous method.
They demonstrated that the "difficulty-based curation" is necessary by running an experiment where they used random problems instead, which resulted in significantly lower scores.
They suggest that this approach could work for any new programming language that emerges, provided you have a compiler and some test cases, but they haven't tested it on every language in existence yet.

In short, the paper suggests that if you want to teach an AI a rare language, don't just throw it into the deep end. First, build a textbook by letting the AI practice offline until it gets it right, teach it the grammar, and then let it train on problems that are just hard enough to make it sweat. It's a cheaper, faster, and smarter way to build code-generating wizards.

Technical Summary: Selective Left-Shift for Low-Resource Code Generation

Problem Statement

Large Language Models (LLMs) exhibit strong code generation capabilities for high-resource languages (HRPLs) like Python and Java but suffer significant performance degradation on Low-Resource Programming Languages (LRPLs) such as Julia and Ballerina. Improving Small Language Models (SLMs) for these languages faces a "trilemma":

Data Scarcity: Supervised Fine-Tuning (SFT) is bottlenecked by the lack of high-quality training data (e.g., CodeNet contains only ~10.8k Julia samples vs. 3.2M for Python).
Inference Cost: Test-time scaling strategies (iterative self-correction, reflection) improve performance but introduce unacceptable latency and computational overhead for real-world deployment.
Sparse Rewards: Applying Reinforcement Learning (RL) directly to base models yields near-zero advantages because the models generate excessive syntax errors, preventing them from receiving the positive feedback necessary to learn algorithmic logic.

Methodology

The authors propose a three-phase pipeline that decouples syntax acquisition from algorithmic reasoning, effectively "left-shifting" expensive inference-time compute into an offline data curation engine.

Phase 1: Offline Data Synthesis via Iterative Feedback

Instead of using iterative refinement during live inference, the pipeline repurposes it as an offline engine to synthesize verified training data.

Process: A base model generates candidate solutions for language-agnostic algorithmic problems. These candidates are executed in a sandbox against IO test suites.
Feedback Loop: If a candidate fails (due to compilation errors or test mismatches), the specific error message or failed test case (expected vs. actual output) is fed back to the model to prompt self-correction.
Verification: This loop continues until a solution passes all tests or a maximum iteration count is reached. Only verified solutions are retained to form the dataset $D_L$ .
Strategy: The system employs a two-tiered verification approach: differential testing (comparing against a reference HRPL solution) when available, and IO-based testing when only problem specifications exist.

Phase 2: Syntax-Aware Supervised Fine-Tuning (SFT)

The base SLM is fine-tuned on the synthesized, verified dataset $D_L$ .

Objective: To embed strong syntactic and grammatical priors specific to the target LRPL (e.g., Julia's 1-based indexing or Ballerina's union types).
Rationale: This phase constrains the model's action space, ensuring that subsequent RL exploration focuses on functional logic rather than rediscovering basic syntax.

Phase 3: Reinforcement Learning with Verifiable Rewards (RLVR)

The SFT-initialized model undergoes RL training using Group Relative Policy Optimization (GRPO).

Data Curation: Training data is curated based on difficulty scores (ELO ratings). The pipeline selects problems at the "edge of capability" where the model has a non-zero probability of success but not guaranteed success, maximizing the learning signal.
Reward Function: Rewards are grounded in language-agnostic IO tests. They are composite, combining a test score (proportional to passed cases) and a build-quality score (penalizing compilation errors).
Zero-Advantage Masking: To prevent the policy from converging on sub-optimal heuristics when all candidates in a group fail, the pipeline sets the advantage to zero if no completion in the group achieves the maximum reward. This ensures updates only occur when there is a meaningful distinction between candidates.

Key Contributions

Three-Phase Pipeline: A novel workflow combining offline inference scaling (data synthesis), syntax-aware SFT, and difficulty-curated GRPO that achieves state-of-the-art results for Julia and Ballerina using an 8B-parameter model.
Left-Shifting Compute: Demonstrates that moving iterative refinement from online inference to offline data synthesis resolves the data scarcity bottleneck without human annotation, turning a recurring inference cost into a one-time investment.
Generalization to Extreme LRPLs: Successfully bootstrapped code generation for Ballerina, a language with near-zero pretraining representation, proving the pipeline's applicability to any language given a compiler and language-agnostic problems.
Benchmark Extension: Extended the MultiPL-E benchmark to include Ballerina, providing a standardized testbed for future research.

Results

Evaluated on Qwen3-8B, the pipeline demonstrated significant improvements over the base model and prior state-of-the-art (SOTA) methods:

Julia Performance:
- MultiPL-E: Improved Pass@1 by +24.6 points (reaching 68.6%), surpassing the previous SOTA (Agnostics) by +7.6 points.
- Ag-LCB (LiveCodeBench): Improved Pass@1 by +30.2 points (reaching 39.2%), surpassing SOTA by +14.2 points.
Ballerina Performance:
- Achieved 49.7% Pass@1 on MultiPL-E and 25.0% on Ag-LCB, representing massive gains from a base model with near-zero prior knowledge.
Efficiency:
- The pipeline used 1/3 of the data and 1/6 of the cost compared to the Agnostics pipeline.
- Total cost was approximately $54.02 compared to $320.3 for the baseline, driven by the shift to offline synthesis and compact, difficulty-filtered RL training.

Significance and Claims

The paper claims that this approach resolves the fundamental trilemma of low-resource code generation. By left-shifting the compute-intensive task of iterative refinement to an offline phase, the authors create a high-quality, verified dataset that enables effective SFT. This SFT prior is critical; it prevents the RL phase from wasting resources on syntax errors, allowing the model to focus on algorithmic reasoning.

The authors emphasize that difficulty-based curation is essential for stable RL training. Training on problems that are too easy or too hard results in collapsed advantage signals (zero variance), whereas selecting problems at the boundary of the model's capability maximizes the gradient signal.

Finally, the paper argues that this pipeline provides a practical recipe for bootstrapping any new programming language. Given a compiler and a set of language-agnostic problems, organizations can generate functional code generation capabilities for niche or emerging languages without relying on massive pretraining corpora or expensive online scaling strategies.