🤖 machine learning

Agentic-imodels: Evolving agentic interpretability tools via autoresearch

This paper introduces Agentic-imodels, an autoresearch framework that evolves scikit-learn-compatible regressors optimized for both predictive accuracy and agent-facing interpretability, demonstrating that models designed to be "simulatable" by LLMs significantly enhance downstream agentic data science performance.

Original authors: Chandan Singh, Yan Shuo Tan, Weijia Xu, Zelalem Gero, Weiwei Yang, Michel Galley, Jianfeng Gao

Published 2026-05-06

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Chandan Singh, Yan Shuo Tan, Weijia Xu, Zelalem Gero, Weiwei Yang, Michel Galley, Jianfeng Gao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a team of super-smart robot assistants (AI agents) whose job is to analyze data, find patterns, and make predictions. Currently, these robots are great at crunching numbers, but they are terrible at reading the "receipts" or explanations left by the tools they use.

Think of it like this: You ask a human data scientist to explain a complex math problem. They might draw a beautiful, colorful graph or write a long, poetic paragraph. A human can read that and say, "Ah, I get it!" But if you hand that same colorful graph or poem to a robot, the robot might get confused. It can't "read" the art; it needs a clear, step-by-step instruction manual written in plain code.

The Problem
Right now, most data science tools are designed to be "human-readable." They produce outputs like fancy charts or complex text that humans love but robots struggle to parse. This creates a bottleneck: as robots take over more data work, they keep tripping over tools they can't understand, leading to mistakes or unreliable results.

The Solution: AGENTIC-IMODELS
The researchers created a new system called AGENTIC-IMODELS. Instead of asking a human to design a tool, they asked a "coding robot" to design a tool specifically for other robots to understand.

Here is how they did it, using a simple analogy:

1. The "Auto-Research" Loop

Imagine a robot chef who is trying to invent a new recipe.

The Goal: The chef wants a dish that tastes amazing (high predictive performance) but is also easy to describe in a single sentence so another robot can cook it exactly the same way (high agent interpretability).
The Process: The chef (a coding AI) writes a recipe, serves it to a taste-tester (the LLM evaluator), and asks, "Can you guess the ingredients and cooking steps just by reading my description?"
The Feedback: If the taste-tester gets it wrong, the chef tweaks the recipe. Maybe they simplify the description or change the ingredients. They repeat this thousands of times, getting better and better at making recipes that are both delicious and easy to explain.

2. The "Simulatability" Test

How do they know if a robot understands the tool? They use a test called simulatability.

They give the robot a description of a model (like a list of numbers and rules).
Then, they ask the robot questions like: "If I change this number, what happens to the result?" or "Which factor matters the most?"
If the robot can answer correctly just by reading the description (without seeing the hidden math), the tool passes the test.

3. The Results

The researchers let their coding robot run this loop for a while. It discovered new types of mathematical models that didn't exist before.

Better than the old stuff: These new models were just as good at predicting the future as the "black box" models (which are accurate but impossible to explain) and much better at being understood by robots than the old "human-friendly" models.
The Pareto Frontier: Imagine a graph where the top-left corner is the "dream zone" (high accuracy + high understanding). The old tools were stuck in the middle or bottom. The new models created by AGENTIC-IMODELS pushed the boundary, filling that dream zone.

4. The Real-World Test

To prove this worked, they gave these new models to four different AI agents (like GitHub Copilot, Claude, and Codex) and asked them to solve real data problems.

The Outcome: When the agents used these new, robot-friendly models, their performance jumped significantly. Some agents improved their scores by up to 73%.
Why? It wasn't just because the agents were told to use a specific package; it was because the models themselves were written in a language the agents could actually "think" through.

Summary

In short, the paper introduces a system where an AI designs data tools specifically for other AIs to use. By forcing the tools to be "simulatable" (easy for a robot to read and predict), they created a new generation of data models that are both highly accurate and perfectly understandable by machines, making the entire process of automated data science much more reliable.

Technical Summary: AGENTIC-IMODELS

Problem Statement

The field of Agentic Data Science (ADS) is rapidly advancing, with systems increasingly delegating intermediate analytical tasks—such as model selection, coefficient interpretation, and conclusion drawing—to autonomous agents. However, a critical misalignment exists: current ADS systems rely on interpretability tools and models designed for human consumption. These tools often produce outputs containing visualizations, complex intervenable components, or unstructured text that are difficult for agents to parse reliably. This mismatch leads to unreliable analysis, obscured analytical choices, and potential failures in the data science lifecycle. While the literature on human-centered interpretable machine learning (e.g., decision trees, generalized additive models) is rich, it has not been adapted to optimize for agent interpretability, defined as the ability of an LLM to accurately simulate a model's behavior solely by reading its string representation.

Methodology

The authors propose AGENTIC-IMODELS, an agentic autoresearch loop designed to evolve data-science tools specifically optimized for agent interpretability. The system operates through the following components:

1. The Autoresearch Loop

The core mechanism is a coding agent (e.g., Claude Code or Codex) that iteratively modifies a Python class (interpretable_regressor.py) to simultaneously maximize two objectives:

Predictive Performance: Measured by the average rank of Root Mean Squared Error (RMSE) across a suite of 65 tabular regression datasets.
Agent Interpretability: Measured by a novel Agent Interpretability Score, defined as the pass rate of LLM-graded tests.

The loop proceeds in steps: the agent proposes a model modification, the model is fitted on training data, its __str__ output is generated, and an LLM evaluator attempts to answer quantitative questions about the model's behavior based only on that string. If the agent's answers match the ground truth (derived from the fitted function), the test is passed.

2. Agent Interpretability Metric

The metric replaces human experiments with automated LLM-based tests. The suite consists of 200 tests divided into six categories:

Feature Attribution: Identifying important features, ranking them, and detecting irrelevant ones.
Point Simulation: Predicting outputs for specific inputs.
Sensitivity Analysis: Determining direction of change and unit sensitivity.
Counterfactual Reasoning: Solving inverse problems (finding inputs for target outputs).
Structural Understanding: Assessing model compactness and decision regions.
Complex Function Simulation: Simulating nonlinearities, interactions, and piecewise functions.

To prevent "reward hacking" (where models simply memorize test answers), the suite is split into a development set (used during optimization) and a held-out set (used for evaluation).

3. Experimental Setup

The authors ran the loop using Claude Code (Opus-4.6/4.7) and Codex (GPT-5.3) across various reasoning effort levels. They evaluated the evolved models against 16 baselines from five families: Linear, Tree, Additive, Rule-based, and Black-box models. The evaluation included held-out datasets and held-out interpretability tests to ensure generalization.

Key Contributions

Reframing Interpretability: The paper shifts the paradigm from designing models interpretable by humans to designing models interpretable by agents. It posits that the "string representation" of a model is the primary interface for agent reasoning.
AGENTIC-IMODELS Framework: A fully autonomous autoresearch loop that discovers new model classes by optimizing for a novel, automated interpretability metric.
New Model Classes: The system discovered a library of scikit-learn-compatible regressors that achieve Pareto improvements over existing baselines, pushing the frontier of predictive performance and agent interpretability.
Downstream Impact: The evolved models were integrated as a Python package and tested in end-to-end ADS workflows, demonstrating significant performance gains.

Results

Pareto Improvements: The evolved AGENTIC-IMODELS occupy a region of the performance-interpretability space previously unoccupied by baselines. For instance, models like HingeEBM achieved a normalized prediction rank of 0.19 (near the best black-box TabPFN) while maintaining an agent interpretability score of 0.71 (significantly higher than TabPFN's 0.17).
Generalization: The models generalized to held-out datasets and held-out interpretability tests. While some models exhibited reward hacking on the development set, the majority showed strong correlation between development and held-out interpretability scores ( $r=0.65$ ).
End-to-End ADS Performance: Equipping four different ADS agents (Copilot CLI, Claude Code, Codex) with the evolved AGENTIC-IMODELS library improved their performance on the BLADE benchmark by 8% to 73% across correctness, completeness, and clarity metrics. The improvements were most pronounced for agents with weaker base models.
Qualitative Patterns: The evolved models often employed "bounded display complexity" (e.g., hard caps on tree depth or feature budgets) and "display optimization" (decoupling the predictive engine from the string representation, such as using a hidden EBM to correct residuals while displaying a simplified linear equation).

Significance and Claims

The authors claim that AGENTIC-IMODELS represents a critical step toward ensuring that insights from data are correctly understood and acted upon by AI agents. As ADS systems become more capable, the tools they use must be designed for the agents themselves rather than human observers.

The paper positions this work as the first to systematically adapt structural properties aiding human interpretability (sparsity, additivity, modularity) into properties suitable for LLMs. The authors note that while the current metric relies on LLM evaluators (which may have biases), the strong correlation between the metric and downstream ADS performance suggests it captures a useful signal. They conclude that as LLMs improve at parsing structured models, frameworks like AGENTIC-IMODELS will enable more sophisticated, potentially superhuman interpretation of complex data, moving the field toward a future where agents conduct the vast majority of data science work.