💬 NLP

Routesplain: Towards Faithful and Intervenable Routing for Software-related Tasks

Routesplain is the first interpretable LLM router for software-related tasks that extracts human-understandable concepts from queries to make faithful routing decisions, outperforming both individual models and black-box baselines in accuracy and cost while enabling concept-level interventions.

Original authors: Adam Štorek, Vikas Upadhyay, Marianne Menglin Liu, Daniel W. Peterson, Anshul Mittal, Sujeeth Bharadwaj, Fahad Shah, Sujith Ravi, Dan Roth

Published 2026-08-11

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Adam Štorek, Vikas Upadhyay, Marianne Menglin Liu, Daniel W. Peterson, Anshul Mittal, Sujeeth Bharadwaj, Fahad Shah, Sujith Ravi, Dan Roth

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are walking into a massive, futuristic library where thousands of different robots are waiting to answer your questions. Some robots are brilliant geniuses who can solve complex math problems but cost a fortune to hire. Others are quick, cheap workers who are great at writing simple stories but might get confused by tricky puzzles. In the world of artificial intelligence, these robots are called "Large Language Models" (LLMs). They are the engines behind the chatbots and code helpers we use today. But here's the catch: not every robot is good at every job. Sometimes, you don't need a genius; you just need a quick, affordable worker. The big question scientists are asking is: How do we know which robot to pick for the specific job you have, without wasting money or getting a bad answer?

This is where the idea of "routing" comes in. Think of routing like a smart librarian who listens to your question, figures out exactly what kind of help you need, and then points you to the perfect robot in the library. For a long time, these librarians were "black boxes." They made decisions based on secret, complicated math that no one could see or understand. If the librarian sent you to the wrong robot, you had no idea why, and you couldn't really fix it. This new paper introduces a new kind of librarian that is different: it doesn't just guess; it explains its thinking in plain language before making a choice.

Meet Routesplain: The Librarian Who Explains Its Choices

The authors of this paper, working with researchers from Columbia University and Oracle AI, have built a new system called Routesplain. Their goal was to solve a very specific problem: how to route software-related questions (like "fix this broken code" or "write a program in Python") to the best AI model, while saving money and making sure the answer is actually correct.

Instead of using a mysterious black box to decide which AI to use, Routesplain acts like a detective that breaks every question down into simple, understandable clues. They call these clues "concepts." When you ask a question, Routesplain doesn't just look at the words; it asks itself a few specific questions:

What is the task? (Is it writing code, fixing a bug, or answering a science question?)
What language is involved? (Is it Python, Rust, or maybe a specific natural language like Spanish?)
How hard is it? (Is this a simple request or a super complex puzzle?)
What domain is it? (Is it about networks, time, or general math?)

Once Routesplain identifies these concepts, it uses a second step to decide which AI model is the best fit. It's like saying, "Oh, this is a tricky Rust programming problem, so we should send it to the Rust-specialist robot, not the cheap generalist one."

Why This Matters: The "Black Box" Problem
Before this paper, most systems that chose AI models were like a magic 8-ball. You asked a question, and it gave an answer, but you couldn't see the logic. The authors argue that this is bad because if the system makes a mistake, you can't fix it. Routesplain is different because it is interpretable. It shows you the concepts it used to make the decision. If you think the system misunderstood the difficulty of the task, you can actually step in and say, "Wait, this isn't hard, it's easy," and the system will instantly change its mind and pick a different, cheaper robot. This is called intervention, and it's a game-changer because it puts humans back in the driver's seat.

The Big Test: 16 Robots, 8 Tasks
To see if their idea worked, the researchers set up a massive test. They gathered 16 of the most advanced AI models available (including big names like GPT-4.1, o3, and Llama 4) and tested them on 8 different types of software tasks. These tasks ranged from writing code and fixing broken programs to answering computer science questions in 29 different languages.

They found something surprising: different models are wildly different at different jobs.

Some models were amazing at fixing code but terrible at writing it.
Some models were great at English but struggled with languages like Hindi or Wolof.
Some models were super expensive but only slightly better than cheap ones for simple tasks.

Because of these differences, simply picking the "smartest" model for everything was a waste of money. Sometimes, a cheap model was actually the best choice.

The Results: Smarter and Cheaper
When they put Routesplain to the test, it did something incredible. It performed just as well as the most advanced "black box" systems (which are hard to understand) but was much better at saving money.

Accuracy: Routesplain matched the performance of the best black-box routers.
Cost: By routing questions to the right model, it saved a lot of money. For example, it could get answers that were 8.5 percentage points more accurate than using just one model, while spending 39% less money.
The Bottleneck: The researchers also used their "explainable" system to find out where things went wrong. They discovered that the biggest mistake the system made was guessing how complex a task was. If the system thought a task was easy when it was actually hard, it picked a cheap model and got a bad answer. This tells future engineers exactly what to fix: make the "complexity detector" smarter.

The Takeaway
The paper suggests that we don't need to rely on mysterious, unexplainable AI to make decisions. By breaking questions down into simple, human-readable concepts like "language," "domain," and "difficulty," we can build routing systems that are not only accurate but also fair, transparent, and editable.

The authors show that we can have our cake and eat it too: we can get the high performance of the smartest AI models while paying the low cost of the cheaper ones, all while knowing exactly why a decision was made. It's like having a librarian who doesn't just point you to a book, but tells you, "I picked this because you asked for a mystery novel, and this is the best mystery novel in the library," and if you say, "No, I actually want a romance," the librarian instantly swaps the book without you having to leave the room.

Technical Summary: Routesplain

Problem Statement

Large Language Models (LLMs) are increasingly deployed for software-related tasks, including code generation, repair, input/output prediction, and computer science QA. However, performance varies significantly across and within these tasks depending on the specific model, programming language, domain, and query complexity. While prior work has explored routing for general-purpose LLMs or simple code completion, existing approaches typically rely on black-box models that lack interpretability. This opacity prevents users from understanding why a specific model was chosen or from intervening in the routing decision. Furthermore, the potential for optimizing both accuracy and cost through fine-grained routing across the diverse landscape of software queries remains underexplored.

Methodology: Routesplain

The authors propose Routesplain, an interpretable LLM router designed specifically for software-related tasks. Unlike black-box routers that map query embeddings directly to model selections, Routesplain decomposes the routing decision into two independently trained stages based on human-interpretable concepts:

Concept Extraction: A concept classifier takes a contextualized query embedding and projects it into a concept space. The concepts are high-level, human-understandable characteristics derived from benchmark metadata and model failure rates, including:
- Task Type: (e.g., code completion, repair, QA).
- Domain: (e.g., computation, network, system).
- Programming Language: (e.g., Python, Rust, Java).
- Natural Language: The language of the query.
- Libraries: Required libraries.
- Complexity: Three measures representing the fraction of models (reasoning, non-reasoning, and total) that failed on a given input.
Model Selection: A separate model classifier takes the predicted concept vector as input and outputs suitability probabilities for a pool of candidate LLMs. This stage is trained with a cost-regularized objective to balance accuracy and token cost.

Key Architectural Features:

Faithfulness: The routing rationale is the set of extracted concepts, which are the exact features used for the decision, avoiding post-hoc explanations that may not reflect actual reasoning.
Intervention: Because concepts are discrete and human-readable, users can modify concept values at inference time (e.g., changing the predicted programming language) to observe routing shifts without retraining.
Diagnostic Transparency: By replacing predicted concepts with ground-truth labels, the system can isolate which concept predictions constitute performance bottlenecks.

Evaluation Setup

The authors evaluated Routesplain on a dataset of 38,685 examples spanning eight software-related tasks:

Datasets: BigCodeBench (Complete, Instruct, Repair, Repair+Info), CRUXEval (Input/Output), MMLU-ProX/Computer Science, and MultiPL-E.
Models: 16 state-of-the-art LLMs from families including GPT-4.1/4o, Grok 3/4, o-series (o1, o3, o4), and Llama 3/4.
Baselines: Individual models, a black-box MLP router, a KNN router, and EmbedLLM.

Key Results

1. Performance and Cost Efficiency

Superiority over Individual Models: Routesplain consistently outperforms all 16 individual models. For instance, at a cost 39.34% lower than GPT-4.1, Routesplain (with cost regularization $\lambda=0.1$ ) achieves 8.52 percentage points (pp) higher accuracy.
Comparison to Black-Box Baselines: Routesplain matches or surpasses the strongest black-box baseline (MLP router) across most cost sensitivity levels. It achieves this while using 15.54% fewer parameters (462K vs. 547K) and providing full interpretability.
Cost-Accuracy Tradeoff: By adjusting the cost regularizer ( $\lambda$ ), Routesplain effectively navigates the Pareto frontier, shifting assignments from expensive models (e.g., o3) to mid-tier (e.g., GPT-4.1 mini) and low-cost models (e.g., GPT-4.1 nano) as cost sensitivity increases.

2. Concept Analysis and Ablation

Concept Importance: The importance of concepts varies with cost sensitivity. At high cost sensitivity ( $\lambda=4.0$ ), complexity concepts have the largest impact on routing decisions. Programming language distinctions remain significant across all cost levels, reflecting distinct model specializations.
Ablation Findings: Removing "complexity" concepts during training causes the most significant drop in accuracy (up to -5.55 pp at high cost sensitivity), indicating that distinguishing between easy and hard queries is critical for cost-effective routing.

3. Intervention and Bottleneck Identification

Complexity as the Primary Bottleneck: Counterfactual experiments revealed that complexity prediction is the primary bottleneck for routing accuracy. Replacing predicted complexity labels with ground-truth oracle labels improved routing accuracy by 3.04–4.05 pp across all cost levels.
Robustness to Other Errors: Errors in predicting other concepts (e.g., programming language, libraries) did not significantly degrade performance when ground-truth labels were substituted, suggesting the second-stage classifier can absorb these errors.
Controllability: Changing a single concept (e.g., programming language) in the input vector predictably shifts routing probabilities. For example, changing the language concept to PHP increased the selection probability of top-performing PHP models (Llama series) by 37.03 pp, validating the system's controllability.

Significance and Claims

The paper claims that Routesplain demonstrates that routing decisions for software tasks can be decomposed entirely into human-interpretable concepts without sacrificing performance compared to black-box alternatives.

Faithful Rationales: It provides intelligible reasons for routing decisions (the concepts themselves) rather than opaque probabilities.
Intervenable: It allows for "what-if" analysis and manual correction of routing decisions at inference time.
Diagnostic Utility: It enables the identification of specific performance bottlenecks (specifically, the difficulty of predicting query complexity) that are impossible to isolate in black-box routers.
Systematic Variations: The work highlights that LLM performance varies systematically not just across tasks, but within tasks based on high-level characteristics like programming language, natural language, and domain, justifying the need for concept-based routing.

The authors conclude that while the approach requires concept labels for training and retraining when model pools change, the retraining cost is negligible (approx. one minute on a single GPU), making the interpretability and control benefits highly practical for software engineering workflows.