Technical Summary: LLM-AutoDP

1. Problem Statement

Large Language Models (LLMs) require domain-specific fine-tuning to excel in specialized fields such as healthcare. However, domain-specific datasets often contain significant noise and low-quality samples, necessitating effective Data Processing (DP). Traditional DP strategies rely on iterative manual analysis and trial-and-error, which incurs high labor costs and poses severe privacy risks when handling sensitive data (e.g., medical records) due to direct human access.

Existing automated approaches, primarily based on AutoML technologies (e.g., Bayesian optimization, evolutionary algorithms), face two critical limitations in the context of LLM fine-tuning:

1. Lack of Semantic Understanding: These algorithms do not comprehend the intrinsic meaning of DP strategies, leading to slow convergence.
2. Computational Inefficiency: They are not tailored for the massive computational overhead associated with LLM data processing and fine-tuning, making exhaustive search impractical.

The core challenge is to achieve automated data processing that generates high-quality strategies without exposing raw sensitive data to humans and while maintaining computational efficiency.

2. Methodology: LLM-AutoDP Framework

The authors propose LLM-AutoDP, a framework that leverages LLMs as agents to automatically generate and optimize data processing strategies. The system operates through two interactive modules in a closed-loop iterative process:

A. Strategy Generation Module

• Agent Role: Pre-trained LLMs act as agents to design DP strategies.
• Prompt Engineering: The system injects domain-specific knowledge via prompt templates, defining an "Operator Pool" consisting of four functional teams:
  1. Data Cleaning: Deduplication, noise removal, length filtering.
  2. Data Generation: Creating missing questions/answers.
  3. Data Optimization: Refining Q&A accuracy and fluency.
  4. Data Selection: Filtering high-quality samples.
• Iterative Process:
  • Initial Round: The agent generates multiple distinct strategy combinations (sequences of operators) based on predefined instructions.
  • Iterative Optimization: The agent receives feedback scores from the evaluation module. Using a Group Relative Comparison mechanism, the agent analyzes the relative performance of different strategy groups. It refines the prompt by injecting these strategies and their scores as in-context information, guiding the LLM to adjust operator selection, sequencing, and the number of strategies for the next round.

B. Strategy Evaluation Module

To ensure the evaluation loop is fast and reliable, the framework applies generated strategies to raw data, fine-tunes a pre-trained model, and evaluates performance on a validation set. To mitigate the high cost of full-scale LLM fine-tuning for every iteration, three acceleration techniques are introduced:

1. Distribution-Preserving Sampling (DPS): Instead of using the full dataset, a subset is sampled that preserves the statistical distribution of the original data (both clean and noisy components). This reduces data volume while maintaining distributional integrity.
2. Processing Target Selection (PTS): A binary classifier (trained via knowledge distillation from a SOTA LLM) identifies "low-quality" samples requiring processing. High-quality samples are skipped during the DP pipeline, saving computation time.
3. Cache-and-Reuse Mechanism (CRM): The system caches processed datasets. When a new strategy shares a prefix with a previously evaluated strategy, the system reuses the cached intermediate result and only applies the new suffix operations, minimizing redundant computation.

3. Key Contributions

1. Novel Framework: Introduction of LLM-AutoDP, the first framework to utilize LLMs as agents for the systematic, automated processing of data specifically for LLM fine-tuning.
2. Iterative In-Context Learning: A mechanism where agents refine strategies based on group relative comparisons and feedback scores, enabling convergence to high-quality pipelines without direct human intervention or raw data access.
3. Efficiency Acceleration: The proposal of three specific techniques (DPS, PTS, CRM) that collectively reduce the total search time by up to 10× without compromising strategy quality.
4. Privacy Preservation: The framework operates in a "black-box" mode where the agent never views the specific content of the raw data, addressing privacy concerns in sensitive domains.

4. Experimental Results

The framework was evaluated on five medical datasets (cMedQA2, CMD, Huatuo-26M-Lite, Medical-O1-Reasoning-SFT, and a small subset Huatuo-100) and one legal dataset (DISC-Law-SFT), using three model architectures (Qwen2.5-7B, Llama3.1-8B, Gemma-2-9B).

• Performance vs. Unprocessed Data: Models fine-tuned on data processed by LLM-AutoDP achieved over 80% win rates against models trained on raw, unprocessed data across most datasets and judges (GPT-4 and Baichuan-M1-14B-Instruct).
• Performance vs. AutoML Baselines: Compared to SELA (an LLM-agent-based AutoML framework using Monte Carlo Tree Search), LLM-AutoDP achieved approximately a 65% win rate.
• Convergence Speed: LLM-AutoDP autonomously terminated after 4–5 rounds in most experiments, whereas baselines like SELA showed slower convergence and performed only marginally better than random search after 5 iterations.
• Efficiency Gains: The acceleration techniques reduced total search time from ~40–50 hours (baseline) to 2.5–9.6 hours, representing a 76%–95% reduction in time cost.
• Robustness: The system demonstrated stability across different agent models (Qwen3-32B vs. DeepSeek-R1-Distill-Llama-70B) and varying initial strategy counts (2, 4, or 6), with performance fluctuations remaining below 1%.

5. Significance and Claims

The paper claims that LLM-AutoDP offers a faster and more effective solution for automated data processing in LLM fine-tuning compared to existing methods. Its significance lies in:

• Bridging the Gap: It fills the research gap of using LLM agents for systematic data processing in the context of LLM training, moving beyond traditional AutoML which lacks semantic understanding of data quality.
• Scalability and Privacy: By automating the process without human access to raw data, it makes fine-tuning feasible for high-privacy domains like healthcare.
• Practical Efficiency: The combination of semantic strategy generation and computational acceleration techniques makes the iterative search process practical for resource-intensive LLM tasks.

The authors conclude that while the method is highly effective, future work will explore extending the framework to multi-modal data and integrating privacy-preserving mechanisms like differential privacy directly into the training process.