💻 computer science

CHORUS: Complementary Experts for High-Coverage Testbench Stimulus Generation

CHORUS is a post-training framework that leverages behaviorally diverse checkpoints and dense-reward reinforcement learning to create complementary expert models, which are then consolidated into a single 4B model that significantly outperforms larger state-of-the-art models in high-coverage hardware verification testbench stimulus generation.

Original authors: Hejia Zhang, Sheng Lu, Zhongming Yu, Chia-Tung Ho, Brucek Khailany, Jishen Zhao

Published 2026-08-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hejia Zhang, Sheng Lu, Zhongming Yu, Chia-Tung Ho, Brucek Khailany, Jishen Zhao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where computers are like giant, incredibly complex LEGO castles. Before engineers can build these castles for real, they have to make sure every single brick fits perfectly and that the whole structure won't collapse when the wind blows. To do this, they write special "test scripts"—like a robot that runs around the castle, shaking walls, opening doors, and checking if everything works. This process is called hardware verification, and it's a huge part of making sure our phones, cars, and computers don't have hidden bugs.

For a long time, people thought the only way to get better at this was to build bigger and bigger "brains" (computer models) to figure out the best tests. But recently, a new kind of smart computer brain, called a Large Language Model (LLM), started helping. These models are great at writing code, but they often need a little push to learn from their mistakes. Instead of just reading a textbook, they get to "run" their code, see if it breaks, and try again. This is like a video game where you don't just read the manual; you play the level, die, and learn how to beat it next time. The big question researchers are asking is: How do we make these models truly masters of testing hardware without just making them infinitely huge and expensive?

Enter CHORUS, a new method that changes the game. Instead of trying to make one giant, super-smart model do everything, the researchers realized that the path to becoming an expert is full of interesting detours. They found that if you train a model in stages, you end up with several different "versions" of the model. Even though these versions end up with similar overall scores, they are actually experts at different things—like one is great at testing the kitchen, while another is a wizard at testing the garage.

The paper shows that by taking these different "expert" versions and combining them, you can create a single, super-powered model that is much better than any of them alone. In fact, this new 4-billion-parameter model (which is relatively small and compact) managed to score 88.0% on a major hardware testing challenge. This is a massive jump, beating a much larger, famous model called DeepSeek-R1 (which has 671 billion parameters) by 13.5 percentage points.

Here's how they did it, using a simple analogy:

The "Staged" Training:
Imagine you are training a group of athletes. Usually, you pick the best one and keep training them until they are perfect. But the CHORUS team did something different. They trained three different athletes through the same rigorous stages. At the end, all three were almost equally fit, but they had different strengths. One was a sprinter, one was a marathon runner, and one was a high jumper.

The "Complementary" Discovery:
The researchers noticed that these athletes didn't just do the same things. When they faced a specific obstacle, the sprinter might fail, but the high jumper would clear it easily. They were "complementary." If you only looked at their average score, you'd think they were the same, but if you looked at which obstacles they could clear, they were totally different.

The "Merging" Magic:
The team then asked: "Can we combine these three athletes into one super-athlete?"

The Simple Mix (Training-Free Merging): They tried just averaging their muscles together. It worked a little bit, making the new athlete better than any single one, but not perfect.
The Smart Coach (Adaptive Distillation): This was the real breakthrough. They created a new student athlete and acted as a coach. For every new obstacle, the coach looked at the three experts and asked, "Who is best at this specific thing?" If the sprinter was best, the student learned from the sprinter. If the high jumper was best, the student learned from them. Crucially, if none of the experts could do it better than the student, the coach skipped that task so the student wouldn't learn bad habits.

The Result:
By using this "Smart Coach" method, the new single model became a master of all obstacles. It didn't just average the skills; it picked the best skill for every single situation. The result is a small, efficient model that outperforms the massive giants of the industry.

The paper explicitly rules out the idea that you just need a bigger model to solve these problems. They showed that even a massive 671-billion-parameter model couldn't reach the same level of performance as their small, smartly trained 4-billion-parameter model. They also found that simply training one model longer didn't help; the key was keeping the different "stages" of training to preserve those unique, complementary strengths.

In short, CHORUS proves that in the world of hardware testing, having a diverse team of specialists and knowing how to combine their unique talents is far more powerful than having one giant, all-knowing brain. It's a reminder that sometimes, the best way to be a genius is to know when to listen to your different inner voices.

Technical Summary: CHORUS

Problem Statement
Hardware verification is a critical component of chip design, specifically the generation of high-coverage testbench stimuli. Unlike standard code generation, the quality of a testbench is determined by a measurable, non-differentiable execution signal: the coverage achieved when the stimulus is run in a hardware simulator. While Large Language Models (LLMs) have advanced code generation through executable feedback, frontier models (even those with hundreds of billions of parameters) struggle with this task. As shown in the paper, increasing model size alone yields diminishing returns, and standard post-training pipelines (Supervised Fine-Tuning followed by Reinforcement Learning) tend to saturate below the achievable performance ceiling.

Methodology
The authors present CHORUS (Complementary Experts for High-Coverage Testbench Stimulus Generation), a post-training framework designed to extract maximum capability from compact, domain-specialized models (specifically 4B parameter models). The framework is built on two core observations and a three-stage pipeline:

Observation of Complementary Strengths: The authors observe that a staged Supervised Fine-Tuning (SFT) curriculum produces checkpoints that, while varying in initial performance, evolve into distinct "experts" when subjected to identical execution-guided Reinforcement Learning (RL). Although these experts achieve comparable aggregate accuracy, they possess distinct task-level strengths, succeeding on different subsets of hardware designs.
The CHORUS Pipeline:
- Stage 1: Independent RL Training: The framework takes intermediate checkpoints from a staged SFT process (specifically from the LLM4Cov pipeline) and applies an identical execution-guided RL procedure (based on DAPO) to each. This transforms the checkpoints into a set of RL experts ( $\pi^0_{RL}, \pi^1_{RL}, \pi^2_{RL}$ ) that are comparable in overall accuracy but complementary in their specific capabilities.
- Stage 2: Consolidation: To integrate these complementary strengths into a single deployable model, CHORUS explores two consolidation methods:
  - Training-Free Merging: Simple weight averaging (Model Soup) or interference-aware merging (TIES, DELLA) of the expert weights.
  - Adaptive Multi-Teacher On-Policy Distillation (OPD): This is the primary method. A student model is trained on its own rollouts but is guided by a pool of teacher experts. Crucially, the system employs reward-gated routing: for each task, the system identifies the best-performing teacher based on execution reward. Distillation occurs only if that teacher outperforms the current student; otherwise, the task is skipped. This prevents the student from learning inferior behaviors and ensures updates only occur where a demonstrably better target exists.

Key Contributions

Identification of Redundancy and Diversity: The paper demonstrates that while later stages of SFT provide little additional benefit once execution-guided RL is applied, the intermediate checkpoints serve as a vital source of diversity. These checkpoints, when independently RL-tuned, become complementary experts rather than redundant variations.
Reward-Gated Distillation: The introduction of an adaptive multi-teacher distillation mechanism that dynamically routes tasks to the best expert and skips tasks where no expert offers an improvement. This approach avoids the "averaging away" of specialization common in static model merging.
Efficiency: The framework achieves state-of-the-art results using a single 4B parameter model, challenging the notion that larger models are strictly necessary for complex hardware verification tasks.

Results
Evaluated on the CVDP-ECov benchmark (83 hardware repositories) under an agentic refinement protocol:

Performance: The final CHORUS model (consolidated via adaptive OPD) achieves 88.0% Pass@1.
Comparison: This result outperforms the 671B-parameter DeepSeek-R1 by 13.5 percentage points and surpasses the previous state-of-the-art (LLM4Cov Stage-2) by a significant margin.
Analysis of Diversity: The "oracle union" of the three independent experts achieves 90.8% Pass@1, indicating a theoretical headroom of ~5 points over the best single expert. The CHORUS consolidation methods successfully recover a substantial portion of this gap, with training-free merging reaching 86.7% and adaptive OPD reaching 88.0%.
Robustness: The results hold across different inference modes (agentic refinement and direct inference) and coverage metrics.

Significance
The paper claims that CHORUS demonstrates that targeted post-training strategies can unlock capabilities in compact models that exceed those of massive frontier models for specialized engineering tasks. By treating intermediate SFT checkpoints as a source of complementary experts rather than just stepping stones to a final model, and by using execution-guided RL to refine and consolidate these experts, the framework achieves high-coverage stimulus generation without the computational cost of scaling model size. The work suggests that the "SFT-to-RL" pipeline's potential is not just in the final checkpoint, but in the diverse landscape of experts it traverses.

Technical Summary: CHORUS

More like this