Scaling Data Difficulty: Improving Coding Models via Reinforcement Learning on Fresh and Challenging Problems

This paper introduces MicroCoder, a high-quality dataset of curated, recent, and challenging competitive programming problems processed through a four-stage framework with automatic difficulty filtering, which significantly boosts coding model performance on unseen hard tasks compared to existing baselines.

The Gist

Imagine you are trying to teach a brilliant but inexperienced apprentice how to solve complex puzzles. You have two choices for your training manual:

1. The "Easy Mode" Book: A massive library containing thousands of puzzles, but 80% of them are simple riddles like "What is 2+2?" or "Name a fruit." The apprentice gets bored, thinks they are a genius, but fails miserably when faced with a real, tricky challenge.
2. The "Hard Mode" Bootcamp: A much smaller, carefully curated book containing only the toughest, most recent, and most interesting puzzles. The apprentice struggles at first, makes many mistakes, but eventually learns to think deeply and solve problems they never thought possible.

This paper is about building that "Hard Mode" Bootcamp for AI coding models.

Here is the story of how the researchers at Microsoft and Cambridge built MicroCoder, a new dataset designed to make AI coders significantly smarter.

The Problem: Too Much "Fluff"

For a long time, the datasets used to train AI coders were like that "Easy Mode" book. They were huge, but they had three big issues:

• Too Easy: They were filled with simple problems that didn't challenge the AI.
• Outdated: They used old problems that the AI had already seen during its initial training, so it was just "memorizing" answers rather than learning.
• Messy: The instructions were inconsistent (some asked for code in one format, others in another), and many problems had broken test cases (like a math problem with a missing number).

The Solution: The "Four-Stage Filter"

The researchers didn't just grab a bunch of problems; they built a sophisticated factory line to process them. Think of it as a high-end coffee roaster that sorts beans to ensure only the best make it into your cup.

1. Collection (Gathering the Beans): They gathered raw problems from all over the internet and various coding competition platforms.
2. Processing (Cleaning and Roasting): They translated everything into English, fixed broken images or formulas, and standardized the instructions so every problem looked the same. They also generated new test cases to ensure the problems were solvable.
3. Filtering (The Difficulty Sorter): This is the magic step. They used a special AI "judge" to rate every problem on a scale of 1 to 5 based on five different factors (like how hard the logic is, how much knowledge you need, etc.).
   • The Analogy: Imagine a bouncer at an exclusive club. The bouncer checks the ID of every problem. If it's too easy (a "1"), it gets kicked out. If it's challenging (a "4" or "5"), it gets in.
4. Verification (The Final Taste Test): Humans and machines double-checked the final list to make sure the problems actually worked and weren't duplicates.

The Result: MicroCoder

The result is MicroCoder, a dataset of about 13,300 problems.

• It is smaller than some other datasets (which have hundreds of thousands of problems), but it is much denser with quality.
• It focuses on fresh problems (so the AI hasn't seen them before) and hard problems.

The Proof: Training the AI

The researchers trained an AI model using MicroCoder and compared it to models trained on the old, massive "Easy Mode" datasets.

• The "3x" Boost: Within just 300 steps of training, the MicroCoder-trained model improved three times faster than the others.
• The "Hard" Wins: The biggest gains were on medium and hard problems. The AI didn't just get better at easy stuff; it learned to tackle the complex logic it previously failed at.
• The "Recency" Factor: Because the problems were new, the AI couldn't cheat by memorizing answers; it had to actually learn how to think.

Why This Matters

Think of it like athletic training. If you only run on a flat, easy treadmill, you won't get very fit. But if you train on a steep, rocky mountain trail, your muscles will grow stronger, and you'll be able to run anywhere.

MicroCoder is that rocky mountain trail for AI. It proves that quality beats quantity. By carefully curating data that is difficult and fresh, we can build AI models that are much more capable at solving real-world, complex coding problems.

In short: Stop feeding the AI a mountain of easy homework. Give it a few hard, fresh challenges, and watch it learn to think like a master programmer.

Technical Summary

Here is a detailed technical summary of the paper "Scaling Data Difficulty: Improving Coding Models via Reinforcement Learning on Fresh and Challenging Problems."

1. Problem Statement

The paper identifies three critical limitations in existing datasets used for training next-generation code generation models:

• Difficulty Imbalance: Current datasets are dominated by simple problems, lacking the challenging tasks necessary to drive model improvement.
• Recency Deficit: Most datasets lack recent problems. Since models are pre-trained on older data, they have less familiarity with recent, inherently harder problems.
• Format Inconsistency & Data Quality: Training corpora mix diverse formats (e.g., function completion vs. input/output) without standardization, leading to execution errors. Furthermore, web-collected data often contains noise, incomplete descriptions, missing test cases, or irrelevant content.

Existing solutions, such as LLM-generated datasets, often suffer from diversity limitations and verification challenges, while manually curated datasets are limited in scale.

2. Methodology

The authors propose a systematic Four-Stage Data Processing Framework and an Automatic Difficulty Filtering mechanism to create the MicroCoder dataset.

A. Four-Stage Data Processing Framework

1. Collection: Aggregates data from diverse sources, including public datasets (TACO, KodCode, DeepCoder) and private web-collected competitive programming problems.
2. Processing (Standardization):
   • Translation: Converts non-English problems to English.
   • Noise Removal: Eliminates missing images, incomplete formulas, ads, and irrelevant links.
   • Test Case Optimization: Uses LLMs to generate comprehensive test cases for problems lacking them or containing noisy web data. For problems with excessive test cases (hundreds), the system selects the 15 longest cases, assuming length correlates with difficulty.
   • Format Unification: Standardizes prompts to a unified format (LiveCodeBench style) to resolve execution logic differences between function completion and input/output styles.
3. Filtering:
   • Hard Requirements: Enforces text-only problems, uniqueness (via overlap identification), and train-test separation (using 16-gram similarity with a 0.22 threshold).
   • Adaptive Selection: Implements difficulty-based filtering using an LLM-powered multidimensional matrix.
4. Verification: Conducts manual validation to ensure problem readability, completeness, and test case accuracy.

B. Automatic Difficulty Filtering (Predict-Calibrate-Select)

To address the lack of reliable difficulty labels, the authors introduce a Predict-Calibrate-Select framework:

• Predict: An LLM (GPT-4O) assesses problem complexity using a Multi-dimensional Difficulty Matrix across five weighted dimensions:
  1. Problem Comprehension Difficulty (PCD)
  2. Knowledge Breadth Requirements (KBR)
  3. Algorithmic Thinking Complexity (ATC) - High Weight (45%)
  4. Implementation Difficulty (ID) - High Weight (35%)
  5. Output Difficulty (OD)
     Scores are averaged over three independent assessments.
• Calibrate: The predicted scores are calibrated against "ground truth" difficulty, defined by the success rate of a model (Qwen-3-4B) attempting the problem four times. This establishes boundaries (e.g., 2.5 and 2.75) to distinguish Easy, Medium, and Hard categories.
• Select: Problems scoring below the calibrated threshold (e.g., < 2.5) are filtered out, retaining only challenging problems suitable for training.

3. Key Contributions

• MicroCoder Dataset: A curated dataset of 13,300 real competitive programming problems from diverse platforms (AIZU, AtCoder, CodeChef, Kattis). It emphasizes recency and difficulty, containing no generated content.
• Systematic Framework: A robust pipeline that transforms raw, noisy web data into a high-quality, standardized corpus suitable for Reinforcement Learning (RL).
• Difficulty-Aware Curation: Demonstrates that filtering based on a multidimensional difficulty matrix effectively removes simplistic problems while preserving those that drive generalization.
• Performance Validation: Shows that training on difficult, fresh data yields significantly better results than training on larger but easier datasets.

4. Experimental Results

The MicroCoder dataset was evaluated using Qwen3-4B-Instruct and Qwen3-14B models trained with GRPO and DAPO (a variant removing KL loss to encourage diversity) algorithms. Benchmarks included LiveCodeBench v6, AtCoder, and LeetCode (strictly unseen problems).

• Training Efficiency: MicroCoder achieved 3x larger performance gains within the first 300 training steps compared to the widely-used DeepCoder dataset of comparable size.
• Overall Performance:
  • Under DAPO, MicroCoder improved overall accuracy by +4.4 points on LiveCodeBench, +6.0 points on LeetCode, and +3.6 points on AtCoder compared to DeepCoder.
  • Relative improvements reached up to 18.7% on LeetCode and 22.0% on Hard problems.
• Difficulty-Specific Gains:
  • Easy Problems: Marginal gains (performance convergence).
  • Medium/Hard Problems: Substantial gains. For instance, on LeetCode Medium, MicroCoder achieved a +40.4% relative improvement over DeepCoder under DAPO.
  • Training Dynamics: Models trained on MicroCoder showed lower critic rewards on the training set (indicating harder problems) but higher test accuracy, validating that challenging problems drive better generalization.
• Model Scaling: The advantage of MicroCoder increased with model size (1.7B to 14B), suggesting that larger models can better leverage recent, difficult data.

5. Significance

This work provides a paradigm shift in dataset creation for code generation:

1. Quality over Quantity: It demonstrates that a smaller, carefully curated dataset of difficult, recent problems outperforms larger datasets filled with easy or outdated content.
2. Difficulty as a Metric: It establishes a reproducible, LLM-based framework for assessing and filtering problem difficulty, moving beyond platform-specific labels which often fail to reflect actual model capabilities.
3. RL Optimization: The results confirm that Reinforcement Learning algorithms (like DAPO) benefit significantly from training on high-difficulty data, leading to more robust and generalizable coding models.
4. Future Directions: The framework suggests extending difficulty-aware curation to other code tasks (e.g., program correction, translation) and developing dynamic difficulty assessment mechanisms that adapt to evolving model capabilities.