🤖 AI

ProfiliTable: Profiling-Driven Tabular Data Processing via Agentic Workflows

ProfiliTable is an autonomous multi-agent framework that leverages dynamic profiling, knowledge-augmented code synthesis, and closed-loop feedback to reliably transform ambiguous user intents into robust, governance-compliant tabular data processing pipelines, outperforming existing baselines in complex multi-step scenarios.

Original authors: Wei Liu, Yang Gu, Xi Yan, Zihan Nan, Beicheng Xu, Keyao Ding, Bin Cui, Wentao Zhang

Published 2026-05-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Wei Liu, Yang Gu, Xi Yan, Zihan Nan, Beicheng Xu, Keyao Ding, Bin Cui, Wentao Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive, messy spreadsheet full of real-world data. Maybe it's sales records with missing prices, customer lists with inconsistent date formats, or inventory logs where "NA" means something different in every column. Your boss gives you a vague instruction: "Fix the currency column and make sure the dates are all standard."

If you ask a standard AI (like a basic Large Language Model) to do this, it might write code that looks perfect on paper but fails in reality. It might guess that "currency" means dollars, when the data actually uses Euros, Yen, or even handwritten notes. It's like a chef trying to cook a recipe without ever tasting the ingredients first.

ProfiliTable is a new system designed to fix this problem. Instead of just guessing, it acts like a team of expert data detectives who work together to understand the data before they try to fix it.

Here is how it works, using simple analogies:

1. The Problem: The "Blind Painter"

Current AI tools often try to paint a picture (write code) without looking at the canvas (the actual data). They rely on general rules. If you tell them to "standardize currency," they might just write a generic script. If the data has weird symbols or unexpected values, the script crashes or produces nonsense. They are "blind" to the specific quirks of your data.

2. The Solution: A Team of Specialized Agents

ProfiliTable doesn't use one big brain; it uses a team of specialized agents that talk to each other in a loop. Think of it as a construction crew where everyone has a specific job:

The Interpreter (The Project Manager):
This agent listens to your vague instruction ("Fix the currency") and figures out exactly what needs to be done. It decides if this is a simple one-step job or a complex multi-step project.
The Profiler (The Detective):
This is the most important new part. Before writing any code, the Profiler goes into the data warehouse and starts investigating. It doesn't just look at the column names; it actually samples the data.
- Analogy: If you tell the Profiler to "fix the currency," it doesn't guess. It pulls out a magnifying glass, looks at the actual values, and says, "Ah, I see 'USD', '€', and 'Yen' mixed with some typos like 'dollars'." It builds a detailed map of what the data actually looks like.
The Generator (The Builder):
Now that the Builder knows exactly what the data looks like (thanks to the Profiler) and has a library of pre-tested tools (like a toolbox of specific "fix currency" or "fix dates" scripts), it writes the code. Because it has the map, it doesn't guess; it builds a solution tailored to the specific mess it found.
The Evaluator & Summarizer (The Inspectors):
Once the code runs, these agents check the result. Did it actually fix the problem?
- Analogy: If the code left some "€" symbols behind, the Summarizer doesn't just say "Error." It investigates why and tells the Builder, "You missed the Euro symbols because you didn't check the second page of the file."
The Loop (The Refinement Cycle):
If the Inspector finds a mistake, the team doesn't give up. They feed that feedback back to the Profiler and the Builder. The Profiler goes back to investigate the specific part that failed, and the Builder tries again with better information. This happens until the data is clean.

3. Why It's Better

The paper claims that this "Dynamic Profiling" approach is a game-changer for three reasons:

It stops guessing: By actively exploring the data first, the system avoids the "syntactically correct but semantically flawed" code that plagues other AI tools.
It handles complexity: Real-world data problems often require many steps (clean, then sort, then merge). ProfiliTable breaks these big problems into small, manageable tasks, solving them one by one.
It's reliable: In their tests, ProfiliTable was the only system that could successfully run code for 100% of the tasks, even the very difficult ones. Other systems often crashed or produced code that couldn't run at all.

The Bottom Line

ProfiliTable changes the way AI handles data. Instead of blindly applying a rule, it acts like a human expert: it looks at the data, understands its unique quirks, plans a specific fix, builds the solution, and checks the work before calling it done. This makes it much more reliable for turning messy, real-world spreadsheets into clean, usable data.

Technical Summary: ProfiliTable

Problem Statement

Table processing—encompassing cleaning, transformation, augmentation, and matching—is a foundational yet highly error-prone stage in real-world data pipelines. While Large Language Models (LLMs) have shown promise in automating these tasks by generating executable code (e.g., pandas or SQL) from natural language instructions, current approaches suffer from significant limitations. Existing methods often struggle with ambiguous instructions, complex multi-step task structures, and a lack of structured feedback. Consequently, they frequently produce code that is syntactically correct but semantically flawed.

Specifically, many existing agents rely on static, rule-based profiling or limited sampling (e.g., inspecting only a few rows) which fails to capture the full range of data values necessary to interpret vague user intents (e.g., "standardize the currency column"). Without active, adaptive exploration of the actual data content, agents cannot construct appropriate mapping tables or handle domain-specific nuances, leading to brittle performance in production environments.

Methodology: ProfiliTable

To address these challenges, the authors propose ProfiliTable, an autonomous multi-agent framework centered on dynamic profiling. Unlike passive metadata consumption, ProfiliTable treats profiling as an active, iterative process of semantic construction that evolves through interaction, feedback, and knowledge integration.

The framework operates as a closed-loop, self-improving pipeline involving several specialized agents:

Interpreter: Parses the user's natural language instruction to determine task complexity (single-step vs. multi-step) and routes the request accordingly.
Profiler: Acts as the semantic grounding module. It employs a ReAct-style (Reason + Act) exploration loop to actively interrogate the raw input tables. Instead of relying solely on schema metadata, it executes lightweight actions (e.g., sampling unique values, computing distributions) to formulate and test hypotheses about data quality and structure. This generates a dynamic profiling summary ( $p^{(t)}$ ) that bridges linguistic intent with tabular reality.
Decomposer: For multi-step tasks, this agent breaks complex instructions into an ordered sequence of atomic subtasks, enabling targeted operator retrieval for each step.
Generator: Synthesizes executable code by grounding the user's intent and the current profiling summary in a Retrieval-Augmented Generation (RAG) framework. It retrieves curated, pre-validated operator templates from a knowledge base to ensure the generated code relies on reliable, domain-specific primitives.
Evaluator: Executes the generated code in a sandboxed environment and computes a task-specific score ( $s^{(t)}$ ) using ground-truth-aligned logic (e.g., F1 score), ensuring that "runnable" code is also "correct."
Summarizer: Operates in a ReAct-style loop over the processed table to validate alignment with the original intent. It formulates verification hypotheses, executes lightweight validation actions, and provides diagnostic insights ( $\tau^{(t)}$ ) regarding unfulfilled subgoals.
Finalizer: Manages the termination of the iterative loop. It selects the best-performing candidate across all iterations if the success threshold is not met, ensuring the workflow returns the most accurate result observed.

The core innovation is the Unified Dynamic Profiling Context ( $P^{(t)}$ ), which iteratively integrates the active profiling summary, retrieved operators, and feedback history to guide subsequent code generation and refinement.

Key Contributions

The paper outlines three primary contributions:

The ProfiliTable Framework: An autonomous multi-agent system that unifies interactive exploration, feedback-driven refinement, and knowledge-augmented synthesis under a dynamic profiling paradigm to overcome the brittleness of one-shot LLM code generation.
A Comprehensive Benchmark: The construction of a benchmark comprising 18 distinct table processing task types, categorized into cleaning, transformation, augmentation, and matching, designed to reflect real-world curation challenges better than existing benchmarks.
Empirical Validation: Extensive experiments demonstrating that ProfiliTable achieves state-of-the-art (SOTA) performance across both single-step and complex multi-step settings, with consistent gains in correctness, completeness, and execution reliability over strong baselines.

Experimental Results

Experiments were conducted using GPT-4o and GPT-5.2 as the underlying models, comparing ProfiliTable against baselines such as MetaGPT, CAMEL, ChatDev, CleanAgent, and DataGovAgent.

Single-Step Tasks: ProfiliTable achieved an Average Task Score (ATS) of 86.82 (GPT-4o) and 89.66 (GPT-5.2), significantly outperforming the second-best baselines (MetaGPT at 56.21 and ChatDev2.0 at 70.02, respectively).
Multi-Step Tasks: ProfiliTable established new SOTA with ATS scores of 80.19 (GPT-4o) and 82.49 (GPT-5.2). Crucially, it was the only method to achieve a 100% Task-wise Runnable Rate (TRR) in multi-step settings, ensuring every task produced an executable output.
Efficiency: Despite the iterative nature of the framework, ProfiliTable demonstrated high cost-efficiency. It utilized the lowest average token count among top performers (e.g., ~24,907 tokens for GPT-5.2 single-step) while maintaining high accuracy, tracing a Pareto-optimal frontier between token cost and solution quality.
Ablation Studies: Removing any core component (Profiler, Feedback, RAG, or Decomposer) led to performance degradation. The Feedback mechanism was identified as the most critical component, with its removal causing the most severe drop in performance, particularly in multi-step scenarios.

Significance and Claims

The authors claim that ProfiliTable highlights the critical role of dynamic profiling in reliably translating ambiguous user intents into robust, governance-compliant table transformations. The paper argues that reliable table processing requires more than just strong LLMs; it necessitates structured, reflective workflows that treat tables as dynamic, semantically rich artifacts.

By shifting from static, rule-driven profiling to an active, hypothesis-driven exploration, ProfiliTable addresses the "brittleness" of current agents. The framework's ability to achieve 100% runnable rates in complex multi-step scenarios positions it as a viable candidate for production deployment, where execution reliability is as important as code correctness. The work suggests a new paradigm of "profiling-driven agency," where iterative, interactive, and failure-aware systems are essential for automating data-centric tasks.