Technical Summary: Can LLMs Clean Up Your Mess? A Survey of Application-Ready Data Preparation with LLMs

1. Problem Statement

Data preparation remains a critical bottleneck in real-world data-centric applications, such as business intelligence (BI), machine learning (ML) training, and data sharing. Despite its foundational role, traditional methods struggle with the increasing volume and heterogeneity of data. The paper identifies three primary sources of data inefficiency:

1. Consistency & Quality Issues: Non-standard formats, noise, and incompleteness.
2. Isolation & Integration Barriers: Disparate systems, entity ambiguity, and schema conflicts.
3. Semantic & Context Limitations: Missing metadata and unlabeled data.

Traditional approaches rely heavily on static rules, manual interventions, or narrowly scoped models, which suffer from four key limitations:

• High Manual Effort: Dependence on fixed rules and domain-specific configurations (e.g., regular expressions) requires significant expertise and maintenance.
• Limited Semantic Awareness: Rule-based systems lack the ability to resolve semantic ambiguities (synonyms, abbreviations) or capture user intent without commonsense knowledge.
• Poor Generalization: Specialized models often fail when applied to out-of-distribution data or different modalities (e.g., text vs. tabular).
• Reliance on Labeled Data: Small-model approaches require expensive, high-quality labeled examples and struggle to integrate diverse contexts.

2. Methodology and Taxonomy

The paper presents a systematic review of LLM-enhanced data preparation, categorizing the field into three core tasks: Data Cleaning, Data Integration, and Data Enrichment. For each task, the authors analyze representative techniques based on their underlying methodologies, ranging from prompt-based approaches to agentic workflows.

A. Data Cleaning

This task aims to transform corrupted or low-quality data into a trustworthy form. The survey identifies four methodological categories:

1. Prompt-Based End-to-End: Uses structured prompts (Instruction-Guided, Reasoning-Enhanced, or Batch Prompting) to guide LLMs in detecting and correcting errors directly.
2. Function-Synthesis-Oriented: Instructs LLMs to generate executable code (e.g., Python functions) or ontological functional dependencies (OFDs) to perform standardization and error repair.
3. Task-Adaptive Fine-Tuning: Fine-tunes LLMs on synthetic noise-augmented datasets or context-enriched data to learn specific error patterns.
4. Hybrid LLM-ML: Combines LLMs with lightweight machine learning models. LLMs act as teachers to generate pseudo-labels or induce interpretable structures (e.g., decision trees) that guide efficient ML detectors.

B. Data Integration

This task focuses on aligning elements across diverse datasets, specifically Entity Matching and Schema Matching.

1. Entity Matching:
   • Prompt-Based: Uses in-context examples, logical rules, and batch clustering to match records.
   • Task-Adaptive: Fine-tunes models using reasoning traces (distillation) or data-centric strategies (hard negative mining).
   • Multi-Model Collaborative: Coordinates multiple models (e.g., a medium-sized LLM for ranking and a stronger LLM for refinement) to balance cost and accuracy.
2. Schema Matching:
   • Prompt-Based: Employs persona specification and Chain-of-Thought (CoT) reasoning to map columns.
   • Retrieval-Enriched: Augments prompts with context from external knowledge graphs or vector retrieval to resolve semantic ambiguity.
   • Model-Optimized: Utilizes specialized table encoders (e.g., TableGPT2) to capture structural semantics.
   • Agent-Guided: Uses agents to orchestrate pipelines, calling tools for retrieval, matching, and verification.

C. Data Enrichment

This task augments datasets with semantic insights, covering Data Annotation and Data Profiling.

1. Data Annotation:
   • Prompt-Based: Uses iterative self-refinement and CoT reasoning to assign labels.
   • RAG-Assisted: Retrieves relevant examples or knowledge graph triples to ground annotations.
   • Hybrid: Distills knowledge from LLMs into smaller models for scalable, cost-effective annotation.
2. Data Profiling:
   • Prompt-Based: Generates metadata, summaries, and hierarchical structures using constrained prompts.
   • RAG-Assisted: Retrieves external evidence (e.g., related tables, documentation) to generate evidence-based profiles.

3. Key Contributions

The paper distinguishes itself from existing surveys through four main contributions:

1. Holistic Task Scope: Unlike prior works limited to specific tasks or modalities, this survey comprehensively covers cleaning, integration, and enrichment across both tabular and text data.
2. Systematic Taxonomy: It proposes a unified taxonomy organizing methods by underlying techniques (Prompting, RAG, Fine-tuning, Agents) rather than coarse categories.
3. Paradigm Shift Analysis: It explicitly traces the evolution from rule-based systems to LLM agent frameworks, summarizing the advantages (semantic reasoning, automation) and limitations (cost, hallucinations) of this shift.
4. Emerging Challenges and Roadmap: It outlines future directions focusing on scalable LLM-data systems, principled designs for reliable agentic workflows, and robust evaluation protocols.

4. Results and Observations

The survey synthesizes findings from hundreds of recent works, highlighting several methodological trends:

• Shift to Hybrid Methods: Recent work moves beyond exclusive LLM inference toward hybrid approaches where LLMs generate programs or transfer reasoning to smaller models to reduce costs.
• Reduced Emphasis on Fine-Tuning: There is a shift away from maintaining heavily fine-tuned, task-specific models toward optimizing input construction (e.g., retrieval augmentation, structured serialization).
• Limited Agentic Implementation: While agent-based orchestration is promising for autonomous workflows, few systems have been fully studied or implemented in practice, indicating a gap in reliable deployment.
• Task-Specific Differences:
  • Cleaning favors hybrid LLM-ML approaches for accuracy.
  • Integration emphasizes multi-model collaboration for scaling.
  • Enrichment integrates retrieval-augmented and hybrid prompting for semantic understanding.

The paper also summarizes representative datasets (e.g., Chicago Food Inspection, abt-buy, DBpedia) and evaluation metrics (Precision, Recall, F1, MRR, ROUGE) used across the field, noting that current benchmarks often struggle to reflect the quality of open-ended enrichment tasks.

5. Significance and Future Directions

The paper argues that LLM-enhanced methods are rapidly becoming a transformative paradigm for data preparation, driven by the need for application-ready data, powerful LLM techniques, and flexible agent infrastructures. However, it remains modest about the current state of the field, acknowledging significant open challenges:

• Data Cleaning: Future work must address global-aware cleaning (capturing dataset-level constraints) and robust error control (preventing hallucination accumulation in agents). Efficient, hierarchical frameworks are needed to scale to large tables.
• Data Integration: The field requires universal integration techniques that rely less on schema descriptions and more on data instance inference, as well as rule-constrained integration that can enforce complex business logic.
• Data Enrichment: There is a need for interactive human-in-the-loop systems, multi-aspect evaluation benchmarks for free-form outputs, and faithful, evidence-grounded generation to ensure trustworthiness.

Ultimately, the paper posits that while LLMs offer unprecedented capabilities in semantic reasoning and automation, realizing their full potential requires principled designs that balance scalability, reliability, and rigorous evaluation.