Technical Summary: Brick - Spatial Capability Routing for the Mixture-of-Models (MoM) Paradigm

1. Problem Statement

The paper addresses the challenge of efficiently routing queries across a heterogeneous pool of Large Language Models (LLMs) at inference time. The core difficulty lies in defining "difficulty" empirically: a query's complexity is not determined by surface features like token count or domain labels, but by the specific capability requirements of the task relative to a model's strengths and weaknesses.

Existing routing strategies rely on superficial routing (domain-based or length/keyword-based), which fails to capture intra-domain variance. For instance, a short query might target an unsolved problem requiring a frontier model, while a long boilerplate prompt might wrap a trivial task solvable by a small open-weight model. This leads to two inefficiencies:

1. Cost Inefficiency: Dispatching easy queries to expensive frontier models (e.g., Kimi 2.6) wastes budget, as local open-weight models (e.g., Qwen 3.5-9B) often suffice.
2. Latency and Accuracy Trade-offs: Cascade routing (probing models from cheapest to most expensive) incurs significant overhead in agentic workflows where a single user request triggers multiple LLM calls, compounding the cost of rejected "cheap" attempts.

The goal is to dispatch each query to the cheapest model that will answer it correctly, balancing cost, latency, and accuracy without relying on surface heuristics.

2. Methodology: The Brick Router

The authors propose Brick, a multimodal router operating within the Mixture-of-Models (MoM) paradigm. Unlike Mixture of Experts (MoE), which activates sub-networks within a single model, MoM dispatches entire queries to distinct, independent pretrained models.

2.1 The MoM Paradigm

Brick utilizes a pool of three heterogeneous models:

• Qwen3.5-9B: A state-of-the-art open-weight model (<10B parameters).
• DeepSeek-V4-Flash: A state-of-the-art open-weight model (<500B parameters).
• Kimi2.6: A frontier open-source contender.

The pool spans different price points (ranging from ~$0.001 to ~$0.03 per call) and capability profiles, allowing the router to exploit capability complementarity (e.g., one model may refuse a query another solves).

2.2 The Six-Dimensional Capability Space

Brick frames routing as a geometric decision in a six-dimensional capability space. The dimensions are:

1. Coding
2. Creative Synthesis
3. Instruction Following
4. Math Reasoning
5. Planning/Agentic
6. World Knowledge

The Pipeline:

1. Capability Estimation: A fine-tuned ModernBERT classifier maps the query to a probability vector $p(x)$ over the six dimensions.
2. Complexity Estimation: A separate classifier estimates a difficulty score $\tau_q$ (blended from "easy/medium/hard" anchors).
3. Skill Matrix Calibration: An offline-calibrated skill matrix $S$ stores the empirical success rates ($s_{m,c}$) of each model $m$ on each capability $c$, derived from a labeled dataset.
4. Geometric Routing Math:
   • Requirement: The query's requirement vector $r_{q,c}$ is derived from $p(x)$ and the difficulty logit $z_q$.
   • Capacity: The model's capacity vector $v_{m,c}$ is derived from $p(x)$ and the model's skill logit $\text{logit}(s_{m,c})$.
   • Residuals: The router calculates asymmetric residuals: under-capacity ($u$, model too weak) and over-capacity ($o$, model overkill).
   • Scoring: A total score $J_m$ is computed as a weighted Euclidean distance of residuals plus a cost penalty:
     $$J_m = \sqrt{\sum_c (u_{m,c}^2 + \lambda o_{m,c}^2)} + \beta a_m$$
     Where $a_m$ is the normalized cost, $\lambda$ penalizes over-capacity, and $\beta$ controls the cost-quality trade-off.
5. Decision: The model with the minimum $J_m$ is selected.

2.3 The Preference Knob ($r$)

Brick introduces a continuous preference knob $r \in [-1, 1]$ that allows operators to slide between a "max-quality" and "max-saving" profile. This knob modulates the four routing scalars ($\mu, b, \beta, \lambda$) via a calibrated power law, turning the cost-quality trade-off into a deploy-time parameter rather than a code change.

3. Key Contributions

1. MoM Formulation: The paper frames whole-model routing across heterogeneous pools as the MoM paradigm, demonstrating that it recovers 24% of the achievable headroom between the best single model and a theoretical three-model oracle.
2. Rejection of Superficial Routing: It empirically shows that domain-based and length/keyword-based routing fail to capture intra-domain difficulty, underperforming even a simple "always-Kimi" baseline.
3. Brick Router: A calibrated MoM router with an explicit capability-vector representation and a user-facing preference knob.
4. Agentic Efficiency: The paper argues that pure single-step routing (like Brick) is structurally superior to cascade routing for agentic workloads, as it avoids compounding the cost of rejected cascade stages on every step of an agent trajectory.

4. Experimental Results

Evaluated on Dataset A (5,504 queries across six capability dimensions):

• Accuracy vs. Cost:

  • Max-Quality Profile ($r=+1$): Brick achieves 76.98% accuracy, beating the best single model (Kimi 2.6 at 75.02%) and all external routers (RouteLLM, FrugalGPT, Cascade Routing). It does so at 28% lower cost ($0.022 vs $0.031 per call).
  • Neutral Profile ($r=0$): Achieves 74.11% accuracy at 4.71× lower cost than always-Kimi, with only a 0.91 percentage point accuracy loss. It outperforms Cascade Routing by +0.71 pp at comparable cost.
  • Low Profile ($r=-0.5$): Achieves 71.62% accuracy, beating FrugalGPT's 69.42% by +2.20 pp at comparable cost.
  • Min-Cost Profile ($r=-1$): Matches the cheapest model (Qwen) at 63.17% accuracy but at 22.15× lower cost than always-Kimi.

• Latency:

  • Median end-to-end latency for Brick (Max profile) drops from 51.2s (always-Kimi) to 22.8s. This is because Brick routes ~31% of queries to the faster DeepSeek-V4-Flash model even in the max-quality setting.

• Oracle Bound:

  • The theoretical ceiling (Three-Model Oracle) is 83.25%. Brick captures a significant portion of the gap between the best single model and this oracle, though a residual 6.27 pp remains unextracted by the current capability features.

5. Significance and Claims

The paper claims that Brick demonstrates the viability of the MoM paradigm as a first-class deployment pattern. Its significance lies in:

• Empirical Validation: Proving that routing based on a calibrated capability space outperforms heuristic routing and single-model baselines.
• Operational Flexibility: Providing a continuous knob for operators to tune the cost-quality trade-off without retraining or code changes.
• Agentic Scalability: Establishing that pure routing is the only design that scales efficiently to agentic workloads, avoiding the linear cost and latency penalties inherent in cascade routing.
• Complementarity: Highlighting that model pools offer complementary error sets (e.g., handling "honest refusals" vs. hallucinations), which a capability-aware router can exploit to improve overall system reliability.

The authors remain modest, acknowledging that the residual gap to the oracle (6.27 pp) represents information the current router does not yet extract, and that future work should focus on Bayesian smoothing for sparse pools and domain-conditioned complexity estimation.