Technical Summary: ComplexConstraints and Beyond: Expert Rubrics for RLVR

Problem Statement

As Large Language Model (LLM) capabilities advance, traditional evaluation benchmarks are increasingly saturated and misaligned with real-world deployment requirements. Current methods, such as programmatic verification (e.g., IFEval), rely on narrow, surface-level constraints that can be checked via scripts but fail to capture nuanced, context-dependent behaviors. Consequently, models can pass benchmarks by avoiding specific characters or formatting rules while producing incoherent content. Furthermore, there is a growing gap between benchmark performance and the ability to follow complex, layered instructions, maintain behavioral constraints over extended interactions, and execute multi-step professional workflows reliably. Existing benchmarks often lack the "construct validity" to distinguish meaningful capability differences among frontier models.

Methodology

The paper proposes a paradigm shift from programmatic verification to expert-curated rubric-based evaluation. This approach involves domain experts decomposing task success into atomic, verifiable criteria that capture both explicit requirements and pragmatic user intent.

Design Principles for Expert Rubrics

The authors articulate five design principles to ensure rubrics are effective for both evaluation and Reinforcement Learning from Verifiable Rewards (RLVR):

1. Maximum Viable Atomicity: Criteria should reflect the prompt's smallest meaningful unit. Over-decomposition can lead to rewarding fundamentally wrong responses that happen to satisfy superficial sub-criteria.
2. Intent-Aware Criterion Design: Criteria must reflect the user's pragmatic intent (the "why") rather than just literal phrasing. This prevents models from learning to privilege surface phrasing over contextual understanding.
3. Three-Category Criterion Taxonomy: Criteria are categorized asymmetrically to shape the reward signal:
   • Primary Intent: Direct requirements; the main source of reward.
   • Extra Credit (Bonus): Elements that enhance experience but are not required; reward is additive, not penalized if missing.
   • Dodged Bullet (Penalty-only): Checks for avoiding likely misconceptions; penalized if violated but not rewarded if satisfied.
4. Iterative LLM-Judge Calibration: Criteria undergo an iterative process where authors draft criteria, test them against reference responses, and refine language based on LLM verifier disagreements to eliminate ambiguity and prevent reward hacking.
5. Domain-Grounded Task Complexity: For agentic tasks, rubrics decompose success across dimensions like completeness, correctness, constraint satisfaction, and format compliance, providing dense, criterion-level feedback.

The COMPLEXCONSTRAINTS Dataset

To instantiate these principles, the authors introduce COMPLEXCONSTRAINTS, a new dataset comprising approximately 1,000 single-turn prompts. Each prompt is paired with 10–40 atomic rubric criteria authored by domain experts. The dataset is designed to occupy the capability frontier, where even the strongest frontier models achieve a perfect task completion rate of less than 17%.

Training Pipeline

The study employs RLVR (Reinforcement Learning from Verifiable Rewards), using the expert rubrics as reward signals.

• Reward Function: The reward is calculated as a weighted sum of satisfied Primary Intent criteria, plus a scaled bonus for Extra Credit criteria, minus a scaled penalty for violated Dodged Bullet criteria.
• Instruction Following: Models (Qwen3-4B and Qwen3-235B) were trained on ~900 examples from COMPLEXCONSTRAINTS using LoRA fine-tuning.
• Agentic Tasks: A model (GLM 4.6) was trained on the CoreCraft enterprise simulation using Group Relative Policy Optimization (GRPO) with rubric-based rewards.

Key Contributions

1. Design Principles: Articulation of five specific principles for constructing high-quality expert rubrics that improve construct validity and serve as effective RL training signals.
2. COMPLEXCONSTRAINTS Dataset: Introduction of a new, expert-curated instruction-following dataset with high constraint density (10–40 criteria per prompt) designed for both evaluation and training.
3. Empirical Validation: Demonstration that expert rubrics function as highly effective training signals, producing substantial, transferable improvements in both instruction following and agentic task execution.

Results

Instruction Following

Training on approximately 1,000 expert-curated examples yielded significant improvements:

• Qwen3-4B: +15.5 percentage points (pp) improvement in mean per-criterion pass rate.
• Qwen3-235B: +12.2 pp improvement.
• Transfer to AdvancedIF: Despite training exclusively on single-turn tasks, models showed substantial gains on the independent AdvancedIF benchmark, including improvements in multi-turn context (+7.1 pp for Qwen3-4B) and system steerability (+12.4 pp).

Agentic Task Execution

Training GLM 4.6 on the CoreCraft enterprise environment for a single epoch resulted in:

• In-Distribution: +11.4 pp improvement on held-out CoreCraft tasks.
• Out-of-Distribution Transfer: Significant gains on benchmarks the model was never trained on:
  • +4.5 pp on BFCL (Parallel function calling).
  • +7.4 pp on τ 2-Bench (Retail customer service).
  • +6.8 pp on Toolathlon (Long-horizon tool use).
• Reliability: The model's Pass@3 (fraction of tasks solved in all three runs) nearly doubled from 9.3% to 17.6% on Toolathlon.

Significance and Claims

The paper claims that expert-authored rubrics serve a dual purpose: they are superior evaluation instruments that offer higher construct validity and better distinguish frontier model capabilities, and they are highly effective training signals for reinforcement learning.

Key claims regarding significance include:

• Data Efficiency: High-quality expert curation allows for substantial capability gains from modest amounts of data (~1,000 examples), outperforming methods that rely on synthetic rubrics.
• Generalization: The competencies learned through rubric-dense training (constraint tracking, workflow decomposition, quality standards) are generalizable across formats (single-turn to multi-turn) and domains (instruction following to agentic tasks).
• Optimal Difficulty: Expert rubrics allow for the calibration of task difficulty to the "productive middle ground," where learning signals are most informative, avoiding the saturation of easy benchmarks or the noise of impossible tasks.
• Measurement vs. Development: The work establishes that investing in expressiveness for evaluation (rich rubrics) pays a "second dividend" by providing the rich, dense reward signals necessary to train frontier models effectively.

The authors conclude that while expert rubric creation requires infrastructure investment, the returns in both measurement quality and training efficiency justify the approach, offering a path forward for developing and assessing next-generation LLM capabilities.