💬 NLP

IR3DE: A Linear Router for Large Language Models

The paper introduces IR3DE, a lightweight, linear Ridge Regression-based router that efficiently and dynamically selects the most appropriate domain-expert Large Language Model for a given prompt, achieving performance comparable to or better than existing baselines without requiring retraining when new models are added.

Original authors: Eros Fanì, Oğuzhan Ersoy

Published 2026-06-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Eros Fanì, Oğuzhan Ersoy

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive library of different experts: a math genius, a coding wizard, a biology professor, and a history buff. You have a million questions to ask, but you don't want to hire the most expensive, all-knowing super-intelligence for every single question. That would be a waste of money and time. Instead, you want a smart receptionist who can quickly look at your question and say, "Hey, this is a math problem, send it to the math genius," or "This is about biology, send it to the professor."

This paper introduces a new, super-fast, and cheap receptionist called IR3DE.

The Problem with Current Receptionists

Right now, the "receptionists" used to route questions to the right AI experts are often too complicated.

The "Heavy" Receptionist: Some current methods use a whole extra AI model just to decide where to send the question. This is like hiring a second, full-time manager just to read the mail and decide who opens it. It's slow and expensive.
The "Privacy" Problem: To train these heavy receptionists, you usually need to gather all the training data from every expert into one big room. If those experts are in different countries or have strict privacy rules, you can't do that.

The IR3DE Solution: The "Linear" Receptionist

The authors propose IR3DE, which is like a simple, linear calculator instead of a complex brain. It doesn't try to "understand" the deep meaning of every word in a complex way. Instead, it uses a mathematical trick called Ridge Regression (think of it as a very efficient way of drawing a straight line through a cloud of dots to find the best fit).

Here is how it works, step-by-step:

The Token Router (The Quick Scan):
When a question comes in, IR3DE breaks it down into small pieces called "tokens" (like individual words or parts of words). It runs these pieces through a simple mathematical filter. This filter has been "taught" by looking at examples of what math questions look like versus what biology questions look like. It doesn't need to see all the data at once; it can learn from small batches, making it great for privacy.
The Sample Route Selector (The Smart Vote):
This is the clever part. The router gives a "confidence score" for every single word in your question.
- The "Noise" Problem: Some words are boring and appear everywhere. For example, the word "the" or "and" appears in math, biology, and history equally. The router is very confused by these words (high "entropy"). If we let these confused words vote on where to send the question, they mess up the decision.
- The Filter: IR3DE ignores the confused words. It only listens to the top-k words that are the most confident.
- The Vote: It takes those confident words and lets them vote. If the words "equation," "solve," and "variable" all vote for "Math," the question goes to the Math expert.

Why This is Cool (The Results)

The authors tested this in three different scenarios:

General Writing (CLM): Predicting the next word in a story.
Large General Writing (CLMlarge): Same thing, but with bigger models and different topics like law and dialogue.
Reasoning: Hard tasks like solving math problems, writing code, or following complex instructions.

The Findings:

Speed and Cost: IR3DE is incredibly fast and cheap because it's just a simple math formula, not a giant AI model.
Performance: Even though it's "dumb" (linear), it performs just as well as the "smart" (complex) receptionists in general writing tasks.
The Reasoning Win: In the hardest reasoning tasks, IR3DE actually beat the other methods. It achieved a score of 98.4%, meaning it routed questions almost perfectly.
Flexibility: If you want to add a new expert (say, a "Music" expert) later, you don't have to rebuild the whole receptionist from scratch. You just give it a few new examples, and it updates instantly.

The Bottom Line

IR3DE is a lightweight, efficient tool that acts as a smart traffic cop for AI models. It uses simple math to filter out the "noisy" words and let the "confident" words decide which expert should handle the job. It saves money, respects privacy, and gets the job done better than the expensive alternatives, especially when the tasks get tricky.

Technical Summary: IR3DE – A Linear Router for Large Language Models

Problem Statement

As the ecosystem of Large Language Models (LLMs) expands to include both foundational generalist models and specialized domain-expert models, the need for efficient inference routing has become critical. Existing routing strategies generally fall into two categories:

Cost-Performance Trade-offs: These methods route queries based on estimated difficulty, sending harder prompts to larger, more capable models and easier ones to smaller models. This approach does not leverage specific domain expertise.
Domain-Accuracy Routing: These methods aim to route queries to the most relevant specialist model (e.g., a coding expert for code generation). However, current solutions typically rely on additional language models (such as DeBERTa or BERT) to classify queries or generate embeddings. This introduces significant computational overhead and requires collecting domain datasets in a single location to train the router, which poses privacy challenges and scalability issues.

There is a need for a routing mechanism that is computationally cheap, fast, capable of handling decentralized data, and able to dynamically add or remove experts without retraining from scratch.

Methodology: IR3DE

The authors propose IR3DE (Ridge Regression-based Router for Domain Experts), a linear router designed to select the most appropriate domain expert for a given prompt. The method consists of two primary components:

1. Token Router (TR)

The Token Router is a linear model based on Ridge Regression. It operates on token embeddings rather than full sentence embeddings or complex classification heads.

Input Processing: An input text $x$ is tokenized and passed through a pre-trained embedding layer $E$ to produce a matrix of embeddings $E(T(x)) \in \mathbb{R}^{T \times h}$ .
Training Objective: The router learns linear weights $W$ by solving a regularized least squares problem. Given a dataset of tokens from various domains with one-hot encoded domain labels $Y$ , the optimal weights are found via the closed-form solution:
$W^* = (E(T(X))^\top E(T(X)) + \lambda I_h)^{-1} E(T(X)) Y$
where $\lambda$ is the Tikhonov regularization parameter.
Decentralization: The formulation allows for the aggregation of statistics (matrix products) across batches or separate nodes. This means domain datasets do not need to be centralized; they can be processed asynchronously, making the approach suitable for privacy-sensitive or distributed environments.
Inference: For a new input, the router computes softmax probabilities for each token: $R(x) = \text{softmax}(E(T(x))W)$ .

2. Sample Route Selector (SRS)

The SRS determines the final routing decision for the entire prompt based on the token-level probabilities generated by the TR.

Entropy Filtering: The router calculates the Shannon entropy for the softmax probabilities of each token. Tokens with high entropy represent uncertainty (likely common tokens like "the" that appear in all domains).
Top-k Selection: The SRS retains only the top- $k$ tokens with the smallest entropy (i.e., the most confident predictions).
Majority Voting: The retained tokens cast votes for their predicted domains. The final routing decision is determined by majority voting among these high-confidence tokens.
Variants: The paper explores variants including IR3DE-all (using all tokens without entropy filtering) and IR3DE-avg (averaging token embeddings before classification), finding that the entropy-based filtering yields the best results for complex tasks.

Key Contributions

Linear Router Architecture: Introduction of IR3DE, a lightweight router utilizing ridge regression and token embeddings, eliminating the need for heavy language model classifiers.
Decentralized Training Capability: The closed-form solution allows for the computation of router weights using aggregated statistics from distributed datasets, avoiding the need to centralize sensitive domain data.
Dynamic Expert Management: The architecture supports the addition or removal of new domain experts without requiring the router to be retrained from scratch.
Entropy-Based Selection: The proposal of an entropy-based filtering mechanism to select the most discriminative tokens for routing, improving performance on complex reasoning tasks compared to using all tokens.

Experimental Results

The authors evaluated IR3DE across three settings: two Causal Language Modeling (CLM) settings (standard and large-scale) and a Reasoning setting involving distinct tasks per domain.

Baselines: Comparisons were made against domain experts, random routing, averaging experts, and state-of-the-art routers including MoDEM (DeBERTa-based), 1NN, and kNN routers (BERT-based).
CLM Settings: In both CLM and CLMlarge settings, IR3DE achieved performance comparable to or surpassing the baselines. Notably, IR3DE-all achieved an average normalized performance of 100.0 in the standard CLM setting, matching the performance of the experts in their own domains.
Reasoning Setting: IR3DE achieved a normalized average performance of 98.4%, outperforming the second-best baseline (kNN router at 97.6%) and significantly surpassing MoDEM variants.
Efficiency: Unlike BERT or DeBERTa-based routers, IR3DE requires only token embeddings and a single matrix inversion (typically ~1k-by-1k), making it significantly cheaper and faster.
Ablation on $k$ : Experiments showed that filtering tokens by entropy (selecting top- $k$ with lowest entropy) is crucial. Using too few tokens results in shallow signals, while using too many introduces noise from uncertain tokens.

Significance and Claims

The paper claims that IR3DE offers a cheap and fast alternative to existing language-model-based routers. Its primary significance lies in its ability to provide high-accuracy domain routing without the computational overhead of additional classifiers or the privacy risks of centralizing training data.

While the authors acknowledge that as a linear model, IR3DE may be less expressive than complex LLM-based routers for queries requiring rich semantic understanding, they demonstrate that it is highly effective for domain classification tasks. The method enables a dynamic ecosystem of LLMs where new experts can be integrated with minimal disruption, addressing a key scalability bottleneck in the deployment of specialized AI models.