Technical Summary: CoDiQ – Test-Time Scaling for Controllable Difficult Question Generation

1. Problem Statement

Large Reasoning Models (LRMs) have demonstrated significant advancements in complex reasoning tasks, particularly in mathematics and coding. However, their progress is bottlenecked by the scarcity of high-quality, challenging training data. Existing automated methods for synthesizing difficult questions face three primary challenges:

1. Generator Capacity Ceiling: Models struggle to generate questions significantly harder than those they can reliably solve, leading to stalled progress.
2. Solvability–Complexity Trade-off: Forcing complexity often results in "fake hard" questions that are logically inconsistent, ill-defined, or unsolvable.
3. Uncontrollable Difficulty: Difficulty is not directly observable or standardized, making it difficult to scale question generation with precise difficulty control without relying on brittle human-in-the-loop pipelines.

The paper aims to scale the synthesis of high-difficulty questions while ensuring they remain well-posed and solvable, moving beyond purely human-driven or coarse-grained automated approaches.

2. Methodology

The authors propose CoDiQ (Controllable Difficult Question Generation), a framework that leverages test-time scaling to achieve fine-grained difficulty control. The methodology consists of four core components:

2.1 Difficulty-Enhancement Strategies

To guide Large Language Models (LLMs) beyond naive prompting (e.g., "make this harder"), the authors design six systematic strategies that serve as cognitive scaffolds:

• Dimensionality & Constraints
• Mathematical Abstraction
• Inverse & Constructive
• State Explosion
• Theorem Disguise
• Edge Case & Rigor Engineering

These strategies ensure that difficulty arises from reasoning depth rather than superficial modifications.

2.2 The CoDiQ Pipeline

The core of the framework is an iterative evolutionary pipeline (Algorithm 1) that transforms a seed question $Q_0$ into a sequence of increasingly difficult variants $\{Q_1, \dots, Q_n\}$ over up to $T_{max}=8$ rounds.

• Iterative Refinement: The model is prompted to "make it more difficult" at each step.
• Hybrid Verification: Two modules regulate the process:
  • Difficulty Estimation: Uses a relative difficulty paradigm combining LLM-Ranking (using Doubao-Seed-1.8) and ValueNetwork Scoring (using Qwen3-8B hidden states) to assign continuous difficulty scores. This avoids saturation effects seen in absolute scoring.
  • Solvability Verification: Uses Qwen3-32B to verify logical consistency and solvability, ensuring only valid instances are retained.
• Termination Criteria: The process stops if difficulty regresses (non-monotonic) or if the question is flagged as unsolvable.

2.3 CoDiQ-Generator (RL Training)

To overcome the generator capacity ceiling, the authors train a specialized generator (CoDiQ-Generator) based on Qwen3-8B using Reinforcement Learning (RL).

• Data Construction: The RL dataset ($D_{RL}$) captures "boundary instances" where the model successfully generates valid questions for several rounds but fails at the next (due to unsolvability or difficulty stagnation).
• Reward Function: A difficulty-aware reward function balances validity guarantees with progressive difficulty scaling. It penalizes invalid outputs and rewards positive difficulty progression ($\Delta(D) > 0$) weighted by confidence.
• Optimization: The model is fine-tuned using the GRPO algorithm within the VeRL framework.

2.4 CoDiQ-Corpus

Using the CoDiQ-Generator, the authors constructed CoDiQ-Corpus, a dataset of 44,000 competition-grade mathematical and coding question sequences. The dataset is stratified by difficulty, derived from the token budget and verification scores.

3. Key Contributions

1. Difficulty-Enhancement Strategies: Six explicit strategies that guide LLMs to inject algorithmic difficulty systematically, enabling the synthesis of questions surpassing zero-shot baselines.
2. Test-Time Scaling Tendency: The identification of a scaling tendency where extended reasoning token budgets boost difficulty but reduce solvability. The paper characterizes the upper bound of a model's capacity to produce valid, high-difficulty questions.
3. CoDiQ-Corpus: A dataset of 44K competition-grade questions. Experiments show that training LRMs on this corpus significantly enhances reasoning performance compared to existing baselines.

4. Experimental Results

4.1 Generation Performance

• Maximum Solvable Difficulty: On the CoDiQ-Bench (200 curated cases), the CoDiQ-Gen-8B model outperformed significantly larger models (e.g., Qwen3-32B) in generating high-complexity instances. It maintained high validity rates across more iteration rounds, accumulating complexity without premature termination.
• Token Correlation: A strong positive correlation was found between reasoning token consumption and difficulty rankings (Pearson $r \approx 0.83-0.85$), confirming token volume as a reliable proxy for difficulty.
• Scaling Tendency: While larger models (GLM-4.6) maintained high solvability, smaller models often suffered "validity collapse" under CoDiQ prompting. However, the RL-aligned CoDiQ-Gen-8B successfully decoupled difficulty scaling from validity loss.

4.2 Corpus Quality and Difficulty

• Difficulty Comparison: Human evaluation and automated ranking indicated that CoDiQ-Corpus questions are significantly more challenging than existing benchmarks like LiveCodeBench and AIME (Average Difficulty Rank: 87.1% vs. ~51.5% for AIME).
• Solvability: Despite the high difficulty, human evaluation confirmed an 82% solvability rate for accepted instances.
• Verifier Paradox: Analysis revealed that some valid problems were rejected by the verifier because they exceeded the verifier's reasoning horizon, highlighting a limitation where the pipeline's upper bound is capped by the verifier's capability.

4.3 Training Effectiveness

• Curriculum Learning: Training models (CoDiQ-Li-4B) on CoDiQ-Corpus using a budget-controlled curriculum (progressive difficulty) yielded superior results on MATH-500 and AIME 2024 compared to standard RL training on original datasets.
• Performance Gains: CoDiQ-L3-4B achieved a 70.6% score on AIME 2024, outperforming the baseline Qwen3-RL-4B (64.3%).

5. Significance and Claims

The paper claims that CoDiQ provides a principled framework for scaling the difficulty of synthetic reasoning data while maintaining logical validity. Its significance lies in:

• Decoupling Complexity from Curation: It offers a method to generate frontier-level supervision data without relying on expensive, purely human-driven pipelines.
• Controllable Scaling: By introducing test-time scaling and hybrid verification, it addresses the "fake hard" problem, ensuring that increased difficulty does not compromise solvability.
• Empirical Validation: The success of the CoDiQ-Corpus in enhancing downstream reasoning capabilities verifies that scaling controlled-difficulty training questions is an effective strategy for improving LRM performance.

The authors acknowledge limitations, specifically the restriction to English math/code tasks, the computational cost of verification, and the "Verifier Paradox," where valid problems exceeding the verifier's capability may be discarded. They open-source the corpus, generator, and implementations to support future research into scaling laws and automated curriculum learning.