🤖 machine learning

iML: Executable, Problem-Grounded, and Broadly Exploratory Code-Driven AutoML

The paper introduces iML, a multi-agent code-driven AutoML framework that ensures executable, problem-grounded, and broadly exploratory solutions through structured blueprints and iterative debugging, achieving state-of-the-art performance and high reliability on diverse machine learning benchmarks.

Original authors: Dat Le, Duc-Cuong Le, Anh-Son Nguyen, Tuan-Dung Bui, Thu-Trang Nguyen, Son Nguyen, Hieu Dinh Vo

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC0 1.0

Original authors: Dat Le, Duc-Cuong Le, Anh-Son Nguyen, Tuan-Dung Bui, Thu-Trang Nguyen, Son Nguyen, Hieu Dinh Vo

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you want to build a complex machine, like a high-performance race car, but you don't know how to be a mechanic. You ask a very smart, well-read but sometimes overconfident assistant (an AI) to write the blueprints and build the car for you.

In the past, these assistants would often write plans that sounded perfect on paper but fell apart the moment you tried to turn the engine on. They might assume the car has a V8 engine when you only have a V6, or they might forget to bolt the wheels on because they were too busy writing a beautiful speech about speed.

This paper introduces 𝑖ML, a new team of AI assistants designed to fix these problems. Instead of one assistant trying to do everything at once, 𝑖ML uses a multi-agent team that works like a highly organized construction crew. Their goal is to build a machine learning model (the "car") that actually runs, is built on the real materials you have, and explores many different ways to win the race.

Here is how 𝑖ML works, broken down into three simple principles:

1. The "Does It Actually Run?" Rule (Executability)

Most AI assistants write code that looks good in text but crashes when you try to run it. 𝑖ML treats running the code as the most important test.

The Analogy: Imagine a chef who doesn't just write a recipe; they actually cook the dish, taste it, and if it's too salty, they fix it before serving it to the customer.
How 𝑖ML does it: The team breaks the job into small parts (preparing the data, building the model). They build each part separately, then try to assemble them. If the parts don't fit (like a square peg in a round hole), the system immediately stops, finds the exact error, and fixes just that piece. It keeps trying and fixing until the whole machine runs smoothly. This ensures that 90% of the time, the final result actually works.

2. The "Look at What You Have" Rule (Problem Grounding)

Sometimes AI assistants make things up. They might guess that your data is clean when it's actually messy, or assume you want to predict sales when you actually want to predict weather.

The Analogy: Imagine a contractor who ignores the blueprints you gave them and the actual dirt on your lot, and instead decides to build a swimming pool because they think you wanted one.
How 𝑖ML does it: Before writing a single line of code, 𝑖ML sends out a "scout" to inspect your actual data. It counts the missing pieces, checks the types of information, and looks up what other experts have done with similar data. It then creates a strict plan (a "blueprint") based on reality, not guesses. This prevents the team from building a model that is logically sound but useless for your specific problem.

3. The "Try Everything" Rule (Broad Exploration)

Old AutoML systems often pick one path and stick to it, like a driver who only knows how to drive on highways. If the road turns to dirt, they get stuck.

The Analogy: Imagine a detective who only checks one suspect. 𝑖ML is like a detective who interviews three different suspects: one who uses traditional methods, one who uses high-tech gadgets, and one who uses a custom-made tool.
How 𝑖ML does it: The team doesn't just pick one way to solve the problem. They run three different strategies in parallel:
1. Traditional methods (reliable, like a sturdy sedan).
2. Pre-trained models (using existing smart tools, like a sports car).
3. Custom neural networks (building a unique, custom engine from scratch).
  They test all three, see which one wins the race, and pick the winner. If one fails, they don't give up; they just switch to the next best option.

The Team Structure

To make this happen, 𝑖ML organizes its AI agents into specific roles:

The Strategist: Reads the problem and inspects the data to create the master blueprint.
The Specialists: Different agents write the code for specific parts (one for cleaning data, one for building the model) so they don't get confused.
The Assembler & Debugger: They put the pieces together, test them, and if something breaks, they act like a mechanic with a diagnostic tool to find and fix the exact broken bolt without rebuilding the whole car.

The Results

The authors tested this system on two sets of difficult challenges (like the famous Kaggle data science competitions).

Reliability: While other systems failed to produce a working result about half the time, 𝑖ML succeeded 90% of the time.
Performance: 𝑖ML didn't just work; it won. It earned "medals" (top rankings) in 45% of the competitions, beating both older automated tools and other AI agents.
Robustness: Even when the instructions were vague or stripped down to the bare minimum (like saying "fix this data" without explaining how), 𝑖ML still managed to succeed 70% of the time by relying on its inspection of the actual data rather than the text instructions.

In short, 𝑖ML is a system that stops guessing, starts testing, and builds a team of specialists to ensure that the final machine learning model is not just a pretty idea, but a working, high-performing reality.

Technical Summary: $i$ ML – Executable, Problem-Grounded, and Broadly Exploratory Code-Driven AutoML

1. Problem Statement

While Automated Machine Learning (AutoML) has democratized access to machine learning, existing systems face significant limitations in flexibility, transparency, and execution reliability. Traditional AutoML tools often operate as rigid "black boxes" restricted to predefined search spaces (e.g., tabular data), lacking the ability to customize pipelines for complex or multimodal problems.

Recent Large Language Model (LLM)-based agents have shifted toward Code-driven AutoML, synthesizing executable code for preprocessing, training, and evaluation. However, current approaches suffer from three critical flaws:

Lack of Executability: Generated code often appears plausible in text but fails during execution due to runtime errors, type mismatches, or unverified assumptions.
Insufficient Problem Grounding: Planning decisions frequently rely on unverified textual assumptions or "hallucinated" data properties rather than empirical analysis of the actual dataset.
Narrow Exploration: Many systems commit prematurely to a single modeling path (e.g., refining a monolithic script or wrapping a fixed backend like AutoGluon), failing to explore diverse, up-to-date solution strategies across traditional ML, pretrained adaptation, and custom neural architectures.

2. Methodology: The $i$ ML Framework

The authors introduce $i$ ML, a multi-agent code-driven AutoML framework designed to satisfy three core requirements: Executability, Problem Grounding, and Broad Exploration. The framework operates through three coordinated phases:

Phase I: Code-Guided Strategic Planning

Instead of direct-to-code prompting, $i$ ML first constructs a problem-grounded strategic blueprint ( $B$ ).

Task Analysis & Profiling: A Task Analyzer extracts semantic constraints, while a Profiling Agent empirically analyzes the raw dataset ( $\mathcal{D}$ ) to generate meta-features ( $F_{meta}$ ), including schema, missingness patterns, and data quality.
Knowledge Retrieval: A Knowledge Retriever acquires domain constraints and preprocessing strategies from external sources.
Blueprint Synthesis: A Guideline Agent synthesizes a structured blueprint specifying preprocessing directives, model selection logic (choosing between Traditional ML, Pretrained Adaptation, or Custom Neural Architectures), training protocols, and evaluation metrics. This blueprint acts as a set of architectural invariants to constrain the search space.

Phase II: Code-Modular Implementation

Specialized coding agents implement components separately under explicit Interface Contracts ( $IC_{\mathcal{T}}$ ).

Preprocessing Coder: Generates a script ( $S_{prep}$ ) to transform raw data into a verified representation ( $\mathcal{D}'$ ) that strictly satisfies the schema required by the modeling stage.
Modeling Coder: Generates a script ( $S_{model}$ ) for training and inference, ensuring mathematical compatibility with the verified $\mathcal{D}'$ .
Contract-Driven Design: This modularity prevents logic entanglement and allows independent development and repair of components.

Phase III: Code-Verifiable Integration

An Assembler Agent integrates $S_{prep}$ and $S_{model}$ into a unified pipeline ( $S$ ).

Dynamic Execution & Verification: The system executes $S_{prep}$ on the data to validate runtime properties (shapes, types) against the interface contract before proceeding to training.
Iterative Debugging: If execution fails, an Error Analyzer localizes the fault (e.g., API mismatches, tensor shape errors) using runtime tracebacks. A Debugging Agent then iteratively repairs the specific component, utilizing web search for current syntax if necessary, until a valid pipeline is obtained.

3. Key Contributions

Novel Design Principles: The paper establishes three criteria for ideal code-driven AutoML: Executability (reliable end-to-end synthesis), Problem Grounding (decisions based on empirical data and retrieved knowledge), and Diverse/Updated Exploration (searching across heterogeneous solution tracks).
The $i$ ML Framework: A multi-agent architecture that operationalizes these principles through empirical profiling, retrieval-backed planning, modular code synthesis, and dynamic verification.
Extensive Experimental Validation: Comprehensive evaluation on MLE-BENCH and a newly introduced $i$ ML-BENCH (covering 16 diverse Kaggle competitions from 2021–2025), demonstrating superior operational reliability and competitive performance.

4. Experimental Results

The authors evaluated $i$ ML against state-of-the-art baselines, including traditional AutoML (AutoGluon) and code-driven agents (MLZERO, MLE-STAR, AutoML-Agent).

Performance on MLE-BENCH (Lite)

Reliability: $i$ ML achieved a 90% valid submission rate, significantly outperforming MLE-STAR (55%) and AutoML-Agent (40%).
Competitive Quality: It secured 45% of tasks with any medal (Gold/Silver/Bronze) and an 80% "Above Median" rate.
Performance Score: The Average Performance Score (APS) was 0.82, representing a 52%–273% improvement over LLM-based baselines.

Performance on $i$ ML-BENCH

Robustness: $i$ ML achieved the highest APS (0.54), nearly tripling the performance of MLZERO (0.19).
Complex Tasks: While baselines like AutoGluon and MLZERO failed (scored 0.00) on complex multimodal tasks (e.g., PetFinder, Multilabel Classification), $i$ ML maintained high scores (0.67 and 0.69, respectively).
Stripped Descriptions: Even when task descriptions were significantly stripped to a generic prompt, $i$ ML maintained a 70% success rate and an APS of 0.63, demonstrating resilience to ambiguous user intent.

Ablation Studies

Executability: Removing dynamic verification or modular integration caused significant drops in valid submission rates and APS, confirming that static checks are insufficient.
Grounding: Removing empirical data profiling reduced the APS from 0.82 to 0.66, proving that data grounding is a primary driver of competitive quality.
Exploration: Restricting $i$ ML to a single modeling track (e.g., only Traditional ML) resulted in performance collapse on non-tabular tasks, validating the necessity of multi-track exploration.

5. Significance and Claims

The paper claims that $i$ ML establishes a robust paradigm for AutoML by bridging the gap between high-level strategic reasoning and empirically verifiable implementation.

Reliability over Plausibility: Unlike previous agents that generate textually plausible but brittle code, $i$ ML enforces executability through a closed-loop process of assembly, execution, diagnosis, and correction.
Empirical Grounding: By anchoring planning in observed data properties and retrieved knowledge, the system avoids hallucinated assumptions that lead to runtime failures.
Human-Competitive Performance: $i$ ML demonstrates that code-driven AutoML can transcend the performance ceilings of "black-box" optimization tools (like AutoGluon) by architecting native, task-specific pipelines.

The authors conclude that $i$ ML represents a transformative step toward reliable, human-competitive autonomous engineering, providing a verifiable foundation for the next generation of ML systems. They acknowledge that while performance on multimodal tasks remains a frontier for future optimization, the framework successfully addresses the core challenges of logic entanglement and execution reliability in current AutoML agents.