🤖 machine learning

From Instance Selection to Fixed-Pool Data Recipe Search for Supervised Fine-Tuning

This paper introduces AutoSelection, a novel framework that reformulates supervised fine-tuning data selection as a fixed-pool recipe search problem, utilizing a two-layer solver to efficiently discover optimal curation recipes from a raw instruction pool without generating new samples, thereby outperforming traditional instance ranking and full-data training across multiple models and tasks.

Original authors: Haodong Wu, Jiahao Zhang, Lijie Hu, Yongqi Zhang

Published 2026-05-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Haodong Wu, Jiahao Zhang, Lijie Hu, Yongqi Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a chef trying to create the perfect soup. You have a massive pantry (the raw data pool) filled with 90,000 different ingredients (instructions). Your goal is to pick the best combination of ingredients to make a soup that tastes amazing (a smart AI model).

The Old Way: The "Top 10" List

Traditionally, chefs (and AI researchers) tried to solve this by tasting every single ingredient individually, giving each one a score, and then just grabbing the top 10 highest-scoring items.

The Problem: Just because an ingredient tastes good on its own doesn't mean it works well in a soup. Maybe you need a little bit of spicy pepper before you add the salt, or maybe you need to remove the watery vegetables before adding the herbs. The old method missed the order and the combination. It treated the soup as a simple list of "best items" rather than a recipe.

The New Idea: The "Recipe Search"

This paper, AutoSelection, suggests a different approach. Instead of just picking the best ingredients, we should search for the best recipe.

Think of a recipe as a set of instructions:

Filter out all the bland vegetables.
Mix the spicy peppers with the sour lemons.
Remove any duplicate spices.
Keep only the top 50% of the remaining mix.

The paper calls this Fixed-Pool Data Recipe Search. The "pantry" stays exactly the same (we don't go buy new ingredients or invent new ones); we just try different ordered sequences of operations to see which one creates the best final dish.

How AutoSelection Works (The "Smart Chef")

Searching for the perfect recipe is expensive. To test a recipe, you have to actually cook the soup (train the AI model) and taste it (run benchmarks). You only have a limited budget to do this (say, 15 tries). You can't just guess randomly; you need a smart strategy.

AutoSelection is a two-layer "Smart Chef" system that helps you find the best recipe without wasting your budget:

The "Cheat Sheet" (Cached Signals): Before you even start cooking, the system looks at the pantry and writes down notes about every ingredient: "This one is spicy," "That one is heavy," "These two are very similar." This way, when testing a new recipe, it doesn't have to re-taste every single ingredient from scratch. It uses these pre-written notes to predict what the recipe might look like.
The "Trial Run" (Warmup): Before committing to a full cooking session, the chef tries three quick, small tests to see if the soup needs to be big, medium, or small. This helps set the stage.
The "Taste Test" (State Vectors): When a recipe is executed, the system doesn't just look at the final score. It looks at the state of the soup: "Did we keep enough ingredients? Is the flavor balanced?" It creates a "state vector" (a summary report) of the selected ingredients.
The "Taste Panel" (The Search Controller):
- Summarizer: Reads the history of past soups and says, "Hey, every time we added salt after the pepper, it tasted great."
- Proposer: Suggests new recipe variations based on that advice (e.g., "Let's try swapping the order of the salt and pepper").
- Ranker: Looks at the new suggestions and uses a "Gaussian Process" (a fancy math guesser) to predict which one is most likely to be good, then picks the single best candidate to actually cook.
- Reseeder: If the chef keeps making the same soup and it's not getting better (stagnation), the system says, "Okay, let's throw out the current idea and try a completely different starting point."

The Results: Did it Work?

The researchers tested this on three different AI models (small and medium-sized) using a 90,000-item instruction pool. They compared AutoSelection against:

Using the whole pantry (Full Data).
Randomly picking recipes.
Just picking the "Top 10" best ingredients (Single-Operator).

The Findings:

Better Soup: AutoSelection created models that were smarter at reasoning tasks than even the models trained on the entire 90,000-item dataset.
Order Matters: They proved that the sequence of steps matters. For example, doing "Step A then Step B" produced a much better result than "Step B then Step A," even if the ingredients were the same.
Stability: The system didn't just get lucky once; it consistently found good recipes across multiple attempts.
Transferability: A recipe that worked well on a small model (1.5B parameters) tended to work well on a larger model (7B parameters), suggesting the "recipe" itself captures something fundamental about good data.

The Bottom Line

This paper argues that making AI smarter isn't just about finding the "best" data points. It's about finding the best process for filtering and mixing that data. By treating data selection as a recipe search rather than a simple ranking list, and by using a smart, budget-aware search engine (AutoSelection) to find that recipe, we can build better AI models with less wasted effort.

What it is NOT:

It does not invent new data or rewrite the instructions.
It does not use AI to generate new training samples.
It does not claim to work for medical diagnosis or specific clinical uses (it focuses strictly on general reasoning benchmarks like math and logic puzzles).

Technical Summary: From Instance Selection to Fixed-Pool Data Recipe Search

Problem Formulation

Current approaches to Supervised Fine-Tuning (SFT) data selection are predominantly formulated as instance-level ranking: scoring individual samples based on heuristics, influence estimates, or learned representations, and retaining a top- $k$ subset. While effective, this view abstracts away the practical reality of SFT dataset assembly, which often involves multi-stage curation workflows including filtering, source mixing, deduplication, and cleaning.

This paper reframes the problem as fixed-pool data recipe search. In this setting:

The raw instruction pool ( $D_0$ ) is fixed.
A library of grounded operators ( $O$ ) exists (e.g., filtering by difficulty, deduplication, mixing).
A candidate "recipe" is an ordered sequence of these operators.
The goal is not to assign a final score to every instance, but to discover an executable recipe that constructs a high-quality selected subset under a limited budget of full SFT evaluations.
Crucially, the method does not generate, rewrite, or augment training samples; it only selects and composes existing data.

This formulation subsumes single-operator top-k selection (viewed as a degenerate length-one recipe) but explicitly optimizes for operator composition, ordering, and the resulting intermediate subset states.

Methodology: AutoSelection

The authors introduce AutoSelection, a two-layer solver designed to navigate the compositional recipe space efficiently under expensive validation constraints.

1. Fixed-Pool Materialization Layer (Cheap Search-Side Reasoning)
To avoid recomputing expensive signals for every candidate, AutoSelection caches grounded signals over the canonicalized raw pool ( $D'_0$ ):

Task-side signals: Benchmark-conditioned activation similarity (following MONA).
Data-side signals: Intrinsic properties like lexical diversity (N-gram entropy) and action-object branching.
Model-side signals: Cached base model responses, including instruction-following difficulty (IFD), varentropy, and sparse-activation statistics.
State Vectors: For any materialized recipe, a state vector $z(r)$ summarizes the realized subset's properties (e.g., retained token ratios, distribution drift, task relevance). This allows the search controller to compare candidate subsets before committing to a full SFT evaluation.

2. Search-Controller Layer (Budget Allocation)
The controller manages the exploration-exploitation trade-off using a budget of $B$ full evaluations:

Warmup Probes: The search begins by probing three distinct data-scale regimes (low, medium, high retention) to establish an initial seed anchor and identify promising retention levels.
Local Refinement: Using a Summarizer, the system converts evaluated history into data-backed hypotheses. A Proposer then generates sibling candidate recipes by performing local edits (inserting, deleting, swapping, or retuning steps) around the current seed.
Ranking: A Ranker selects the single best candidate for full evaluation. It combines:
- The recipe structure.
- The realized state vector (subset properties).
- A Gaussian Process (GP) surrogate prior based on recipe encodings.
- Historical search data.
Stagnation-Triggered Reseeding: If the search fails to improve the incumbent best score for $P$ consecutive steps, a Reseeder refreshes the seed anchor by selecting a new promising motif from history, preventing the search from getting stuck in saturated local regions.

Language models (specifically DeepSeek-R1) are used strictly as search-side assistants to summarize history, propose grounded edits, rank candidates, and reseed; they do not rewrite or generate training data.

Key Contributions

Problem Formulation: The authors formalize SFT data curation as a budgeted black-box optimization problem over executable recipes on a fixed raw pool, making operator composition and ordering explicit optimization targets.
Method (AutoSelection): A two-layer solver that decouples cached candidate materialization from expensive full SFT evaluation, utilizing warmup probes, state-aware local edits, GP-assisted ranking, and stagnation-triggered reseeding.
Empirical Evidence: Comprehensive evaluation across three base models (Qwen2.5-1.5B/3B, Llama3.2-1B) on a 90K instruction pool.

Experimental Results

Evaluated on a 90K-sample instruction pool with a budget of 15 full SFT evaluations:

Performance: AutoSelection achieved the strongest in-distribution reasoning average across all three base models, outperforming full-data training, random recipe search, random top-k, and single-operator selectors.
- Improvements over full-data training: +1.82 (Llama3.2-1B), +3.40 (Qwen2.5-1.5B), and +3.72 (Qwen2.5-3B) points.
- Improvements over the strongest non-AutoSelection baseline: +0.90 to +2.89 points.
OOD Generalization: On held-out graph-reasoning benchmarks (GraphWiz, NLGraph), AutoSelection remained competitive, ranking first on Qwen2.5-1.5B and nearly tying the best baseline on Qwen2.5-3B.
Stability: Three independent runs on the 1.5B model showed a narrow score range (0.59 points), indicating the gains are not due to a single lucky run.
Ablations:
- Removing Warmup caused the search to spend early evaluations in low-scoring regions.
- Removing the Summarizer resulted in a smoother trajectory but a weaker final best recipe.
- Removing Reseeding led to a flatter best-so-far curve, indicating difficulty escaping local optima.
- Removing State Vectors exposed the search to candidates with promising forms but weak realized subset states.
Structural Sensitivity: Matched-seed structural ablations confirmed that operator ordering and composition significantly impact performance beyond the mere presence of specific operators.
Transferability: A 1.5B-to-7B transfer check showed a positive Spearman rank correlation ( $\rho=0.82$ ), suggesting that high-quality data motifs discovered on smaller models can be useful for larger models, though exact rank preservation is not guaranteed.

Significance and Claims

The paper claims that the value of AutoSelection lies not in discovering a universally best single selector, but in allocating a small number of full evaluations to find a stronger composition of grounded curation decisions.

The authors argue that:

Recipe Structure Matters: The ordering and interaction of operators (e.g., filtering before deduplication vs. after) have a measurable impact on downstream SFT performance, supporting the view of data recipes as first-class optimization objects.
Fixed-Pool Control: By restricting the search to a fixed raw pool without generation or rewriting, the method isolates the effects of operator choices from confounds introduced by data synthesis or prompt engineering.
Efficiency: The approach demonstrates that efficient search strategies (warmup, state-aware ranking, reseeding) can yield better subsets than full-data training or random search within a strict evaluation budget.

The work positions itself as a controlled alternative to LLM-driven data generation systems, focusing on the optimization of existing data workflows rather than the synthesis of new data.