From Insight to Action: A Novel Framework for Interpretability-Guided Data Selection in Large Language Models

1. Problem Statement

While Mechanistic Interpretability tools, particularly Sparse Autoencoders (SAEs), have successfully uncovered meaningful, human-understandable features within Large Language Models (LLMs), a critical gap remains: translating these insights into practical actions for model optimization.

• Current Limitation: Most interpretability research stops at explanation (describing how a model works) rather than prescription (using that knowledge to improve the model).
• The Gap: Existing data selection methods rely on external signals (e.g., perplexity, diversity metrics) or treat the model as a black box. They fail to leverage the model's own internal causal mechanisms to select the most effective training data.
• Goal: To bridge the gap between "Insight" (understanding internal features) and "Action" (optimizing the model) by hypothesizing that data selection guided by a model's internal, causally-validated task features is a highly effective training strategy.

2. Methodology: Interpretability-Guided Data Selection (IGDS)

The authors propose IGDS, a framework that operationalizes interpretability into a two-stage pipeline (termed Insight2Action).

Stage 1: Task Feature Identification

The goal is to isolate features that are not just correlated with, but causally linked to task performance.

1. High-Frequency Recalling:
   • Using an SAE, the model processes a small set of task-related samples.
   • Features are filtered based on activation frequency at critical token positions (e.g., the final token of a prompt).
   • Only features activating above a threshold (e.g., 80%) are retained as candidates.
2. Interventional Filtering (Causal Validation):
   • To distinguish true causal drivers from noise, the authors perform causal interventions.
   • For each candidate feature, its influence vector is added to the model's residual stream to "amplify" the feature.
   • The performance gain ($\Delta$) is measured on a validation set.
   • Only features showing a significantly positive performance gain are selected as the final set of Task Features ($F_{task}$).

Stage 2: Feature-Based Data Scoring and Selection

Once $F_{task}$ is identified, the framework selects training data that maximally activates these specific mechanisms.

1. Feature-Resonant Score (FRS):
   • For every candidate data point in a large pool, the activation magnitudes of all $F_{task}$ features are aggregated at the critical token position.
   • $S(x) = \sum_{f \in F_{task}} a_f(x, t^*)$
2. Selection:
   • Data points are ranked by their FRS.
   • The top-K% of data (e.g., top 50%) are selected to form the Feature-Resonant Data subset for Supervised Fine-Tuning (SFT).

3. Key Contributions

1. Novel Optimization Strategy: Proposes using causally-validated internal features to guide data selection, offering a direct path to efficient capability enhancement.
2. IGDS Framework: Introduces a general, practical pipeline that transforms descriptive interpretability insights into a prescriptive data selection process, closing the loop from analysis to optimization.
3. Empirical Validation: Demonstrates that IGDS outperforms both competitive baselines (quality/diversity-based) and full-dataset fine-tuning while using significantly less data.

4. Experimental Results

The framework was validated on Mathematical Reasoning, Summarization, and Translation tasks across three model families: Gemma-2, LLaMA-3.1, and Qwen3.

• Superior Data Efficiency:
  • Gemma-2-2B (Math Task): IGDS surpassed full-dataset fine-tuning by a remarkable 17.4% in accuracy (pass@8) while using only 50% of the data.
  • Consistency: IGDS consistently outperformed baselines like IFD (Instruction Following Difficulty), Loss-based filtering, ZIP (Diversity), and Random selection across all models and tasks.
• Causal Correlation:
  • Analysis confirmed a strong positive correlation between the amplification of identified features and downstream task performance.
  • Models trained with IGDS showed the highest median feature activation and the best performance scores.
• Robustness:
  • Sampling Rates: IGDS maintained superiority over full-dataset tuning even at low sampling rates (20%, 50%, 80%).
  • Stability: The top-ranked features (e.g., $F_{14,11575}$ for Math) remained consistent across different source datasets (GSM8K, Math500, OpenR1), proving they are intrinsic to the model's mechanism rather than dataset artifacts.
• Computational Efficiency:
  • IGDS is highly efficient, reducing computational costs by ~20% compared to Loss-based selection because feature scoring is integrated into the single forward pass required for gradient calculation.

5. Significance and Conclusion

• Paradigm Shift: This work moves interpretability from a passive analytical tool to an active optimization engine. It proves that "listening" to a model's internal causal mechanisms yields better training data than external heuristics.
• Mechanistic Evidence: The study provides strong evidence that specific, sparse features in LLMs are causally responsible for task performance, and amplifying them via targeted data selection leads to rapid capability gains.
• Practical Impact: IGDS offers a cost-effective solution for training high-performance models with limited data resources, challenging the notion that "more data" is always better.
• Limitations: The method's efficacy is currently dependent on the quality and coverage of available SAEs. Models with less mature SAE ecosystems (e.g., Qwen3 in this study) showed more modest gains, highlighting the need for broader SAE development.

In summary, IGDS successfully bridges the gap between mechanistic understanding and model improvement, demonstrating that the most potent signals for data selection reside within the model's own internal causal mechanisms.