CHIMERA: Compact Synthetic Data for Generalizable LLM Reasoning

1. Problem Statement

The paper addresses three fundamental data-centric challenges hindering the scalable development of open, general-purpose Large Language Models (LLMs) with advanced reasoning capabilities:

1. Cold-Start Data Scarcity: Effective reasoning post-training requires seed datasets with detailed, long Chain-of-Thought (CoT) trajectories. Existing datasets often lack these long-horizon reasoning steps, providing only ground-truth answers or brief explanations, which are insufficient for bootstrapping complex reasoning policies.
2. Limited Domain Coverage: Most open-source reasoning datasets are heavily concentrated on mathematics and coding. This narrow focus prevents models from generalizing reasoning strategies to broader scientific disciplines (e.g., physics, chemistry, biology, humanities).
3. The Annotation Bottleneck: Frontier-level reasoning tasks are too difficult for reliable, cost-effective human annotation. Generating high-quality CoT explanations for expert-level questions requires deep domain expertise, making large-scale human-curated data prohibitively expensive and slow.

2. Methodology: The CHIMERA Pipeline

The authors introduce CHIMERA, a compact synthetic dataset of 9,225 high-quality samples. The dataset is constructed via a fully automated, three-stage LLM-driven pipeline designed to ensure quality, diversity, and scalability without human intervention.

Stage 1: Subject Expansion

• Input: A small set of high-level seed subjects (e.g., Mathematics, Physics, Computer Science, Chemistry, Biology, Linguistics, History, Literature).
• Process: A strong LLM (GPT-5) is prompted to expand these seeds into a comprehensive, hierarchical taxonomy of fine-grained topics.
• Output: A structured list of over 1,179 unique topics across 8 major disciplines. This ensures broad domain coverage and systematic organization.

Stage 2: Problem Generation

• Process: For each topic, GPT-5 generates self-contained, PhD-level reasoning problems with verifiable answers.
• Constraints: Problems must be solvable by experts, unambiguous, and have a single correct answer.
• Verification (Cross-Model): Two independent LLMs (GPT-5 and o4-mini) act as verifiers to check problem validity and answer correctness. A problem is retained only if both models agree it is well-posed and the answer is correct. This dual-verifier approach mitigates model bias and hallucination.

Stage 3: Solution Synthesis

• Process: A state-of-the-art open reasoning model (Qwen3-235B-A22B-Thinking-2507) generates detailed, step-by-step CoT reasoning trajectories for the validated problems.
• Labeling: The generated trajectory is compared against the ground-truth answer.
  • Correct Trajectories: Used for Supervised Fine-Tuning (SFT).
  • Incorrect Trajectories: Retained as problem-answer pairs for Reinforcement Learning (RL), where only the final answer correctness matters.
• Quality Control: The pipeline employs a fully automated evaluation protocol using strong models to cross-validate, eliminating the need for human annotators.

3. Key Contributions

1. Formalization of Data Challenges: The paper identifies and formalizes the specific data bottlenecks (cold-start, domain coverage, annotation cost) limiting open reasoning models.
2. CHIMERA Dataset: Introduction of a compact (9K samples), fully synthetic dataset featuring:
   • Long CoT Trajectories: Solutions average ~11,000 words, significantly longer than existing datasets, encouraging rigorous multi-step reasoning.
   • Broad Coverage: Spanning 8 scientific/humanistic disciplines and 1,179 fine-grained topics.
   • Automated Quality: A robust, human-free evaluation pipeline ensuring data validity.
3. Demonstration of Data Efficiency: Empirical evidence showing that a carefully constructed, compact synthetic dataset can outperform massive, noisy datasets.

4. Experimental Results

The authors evaluated the effectiveness of CHIMERA by post-training a Qwen3-4B model using a combination of SFT and Reinforcement Learning (CISPO).

• Benchmark Performance: The fine-tuned 4B model achieved strong performance on challenging benchmarks including GPQA-Diamond, AIME 24/25/26, HMMT 25, and Humanity's Last Exam (HLE).
• Scale Comparison: Despite being only 4B parameters, the CHIMERA-tuned model:
  • Matches or surpasses 8B–70B scale baselines (e.g., DeepSeek-R1-Distill-Llama-70B).
  • Approaches the performance of models two orders of magnitude larger (e.g., DeepSeek-R1 and Qwen3-235B).
• Inference-Time Scaling: The model showed consistent gains in pass@k metrics as the sampling budget increased, indicating improved reasoning robustness and solution discovery capabilities.
• Ablation Studies:
  • SFT vs. RL: SFT alone accounted for the majority of performance gains, with RL providing incremental improvements.
  • Difficulty Analysis: The base model achieved near-saturation accuracy (~88%) on other synthetic datasets (e.g., OpenScience, DeepMath), but only ~37.5% on CHIMERA, proving CHIMERA's superior difficulty and training signal.
  • Quality Validation: Blind evaluations by LLM judges rated CHIMERA problems as comparable to or better than human-curated problems (HLE) in terms of clarity and reasoning depth.
  • Contamination Check: N-gram overlap analysis confirmed near-zero leakage between the training data and evaluation benchmarks.

5. Significance and Conclusion

The paper demonstrates that data quality and structure are more critical than sheer scale for advancing LLM reasoning.

• Paradigm Shift: It challenges the notion that massive human-curated datasets are necessary for frontier reasoning, proving that compact, high-quality synthetic data generated by strong models is a viable, scalable, and cost-effective alternative.
• Generalizability: By covering diverse disciplines beyond math, CHIMERA enables models to generalize reasoning strategies across domains, addressing a key limitation of current open-source models.
• Future Impact: The work provides a blueprint for constructing synthetic training data that can unlock high-level reasoning capabilities in smaller, more accessible models, democratizing access to frontier AI capabilities.