ScaleBox: Enabling High-Fidelity and Scalable Code Verification for Large Language Models

1. Problem Statement

The paper identifies two critical bottlenecks in current Large Language Model (LLM) code training and evaluation systems, particularly those utilizing Reinforcement Learning from Verifiable Rewards (RLVR):

• Verification Accuracy Gap (The "Exact Match" Trap): Existing sandboxes predominantly rely on Exact Match (EM) or heuristic string matching to verify code correctness. This approach fails for tasks with multiple valid solutions (non-deterministic outputs) or those requiring numerical precision (floating-point comparisons).
  • Evidence: An analysis of 34,757 problems revealed that 14.57% require specialized verification (Special Judges). Crucially, 59.01% of logically correct ground-truth solutions are falsely rejected by EM, injecting significant noise into RL reward signals and causing biased gradients.
• Scalability and Efficiency Bottlenecks: Current systems often use static deployment and coarse-grained task scheduling. This leads to a host-accelerator imbalance, where GPUs/NPUs are heavily utilized while host CPUs remain underused for parallel code execution. This limits throughput in large-scale, multi-node clusters.

2. Methodology: The ScaleBox System

ScaleBox is a high-fidelity, scalable sandbox infrastructure designed to address these issues. Its architecture consists of three synergistic components:

A. Automated Special Judge Synthesis & Management

To solve the accuracy gap, ScaleBox introduces a pipeline to automatically generate and manage Special Judges (SPJ):

1. Taxonomy Detection: An LLM classifies problems to identify those where EM is insufficient (e.g., non-deterministic paths or numerical tolerance requirements).
2. Program Synthesis: An LLM synthesizes Python-based judge programs (validate_solution) that read inputs, reference outputs, and participant outputs to produce a binary verdict based on custom logic.
3. Verification Loop: Synthesized judges undergo a pre-deployment validation stage (fidelity and robustness tests) within the sandbox to ensure they accept ground-truth solutions and reject incorrect ones before being used in training.

B. Hybrid Parallelism & Distributed Deployment

To solve the efficiency bottleneck, ScaleBox implements a distributed architecture:

• Hybrid Parallelism: Unlike systems that only parallelize at the instance level, ScaleBox parallelizes at both the instance level (multiple LLM requests) and the unit-test level (multiple test cases within a single request) on worker nodes.
• Resource Optimization: It leverages idle CPU resources for code execution and testing, balancing the load between host processors and accelerators (GPUs/NPUs).
• Management: A centralized web-based dashboard allows for hot-updates, real-time resource monitoring, and seamless multi-node coordination via an NGINX load balancer.

C. Configuration-Driven Evaluation Suite

ScaleBox provides a unified framework supporting major code benchmarks (e.g., LiveCodeBench, HumanEval, AetherCode) with reproducible workflows, including standardized test case dispatch and multi-stage verification (optimizing EM first, then falling back to SPJ).

3. Key Contributions

1. ScaleBox System: A high-fidelity, scalable sandbox system specifically engineered for large-scale code RLVR training, featuring automated SPJ synthesis and hybrid parallel execution.
2. Automated SPJ Pipeline: A novel method to synthesize and validate specialized judges, significantly reducing reward noise caused by false negatives in exact-match evaluation.
3. Efficient Infrastructure: A distributed architecture that achieves high throughput by utilizing hybrid parallelism and balancing host-accelerator workloads.
4. Comprehensive Evaluation: A configuration-driven suite for reproducible benchmarking across standard code datasets.

4. Experimental Results

A. Verification Accuracy

• Fidelity: On 27 complex AetherCode problems, synthesized special judges achieved a True Positive Rate (TPR) ≥ 90.0% and True Negative Rate (TNR) ≥ 84.0% across various LLMs (including Qwen3-235B and Claude-3.7-Sonnet).
• Impact: This confirms that automated synthesis can reliably distinguish correct from incorrect code, even for elite competitive programming tasks.

B. Efficiency & Scalability

• Throughput: On a single node (Intel Xeon 64 cores), ScaleBox achieved 39.31 tasks/s, representing a 1.59× speedup over the verl framework and 2.63× over SandboxFusion.
• Scaling: The system scales linearly to 62.10 tasks/s on a 3-node cluster, demonstrating effective utilization of multi-node resources.

C. RL Training Performance (RLVR)

Experiments using Qwen3-8B with the GRPO algorithm showed:

• Performance Gains: On the 1.2K SPJ subset, incorporating special judges improved Pass@1 on LiveCodeBench (LCB-v5) by +5.91% (from 27.24% to 33.15%).
• Generalization: Even on the full 26K dataset (where SPJ cases are only ~10%), performance on LCB-v6 improved by +1.91%.
• Stability: Training curves showed that high-fidelity rewards reduced variance in credit assignment, leading to more stable convergence and preventing the "reward suppression" seen with exact-match baselines.

5. Significance

ScaleBox addresses a fundamental limitation in the current LLM code intelligence ecosystem: the reliance on brittle, exact-match verification. By providing a reliable, high-throughput infrastructure that supports specialized verification logic, ScaleBox enables:

• More Effective RL Training: It allows models to learn from complex, non-deterministic problems that were previously untrainable or noisy due to false negatives.
• Scalable Research: It removes the computational bottleneck of code execution, facilitating larger-scale RL experiments and more robust benchmarking.
• Foundation for Future Agents: It lays the groundwork for advanced code agents in automated software engineering and multi-turn problem solving, where precise feedback is critical.

In summary, ScaleBox bridges the gap between the need for high-fidelity reward signals in RL and the practical constraints of scalable code execution, significantly advancing the state of the art in LLM-based code generation.