AgenticData: An Agentic Data Analytics System for Heterogeneous Data
AgenticData is an innovative agentic system that enables autonomous analysis of heterogeneous structured and unstructured data through natural language queries by leveraging a multi-agent collaboration strategy with feedback-driven planning and semantic optimization to achieve superior accuracy over existing methods.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are trying to solve a massive mystery, but the clues are scattered everywhere. Some clues are neatly written in ledgers with clear columns like "Name," "Date," and "Amount." Others are messy, buried inside thousands of unorganized diary entries, scanned documents, and web pages where the information is hidden in paragraphs of text. For a long time, computers have been great at reading the neat ledgers but terrible at making sense of the messy diaries. They usually need a human expert to write complex instructions (code) to connect the dots between the two. This is like asking a librarian to find a specific fact, but you have to write the entire search engine's code yourself before they can even start looking. It's slow, expensive, and frustrating. The big question in this corner of computer science is: Can we build a system that acts like a super-smart detective, capable of reading both the neat ledgers and the messy diaries, understanding what we want in plain English, and solving the mystery all by itself?
Enter AgenticData, a new system designed to be that super-smart detective. Think of it as a team of specialized agents working together in a bustling newsroom. Instead of one giant brain trying to do everything at once, AgenticData uses a "multi-agent" approach. It has a Data Profiler who scans the entire library to understand what books and files exist and how they might be related. It has a Planner who figures out the step-by-step strategy to answer a question. It has a Validator who acts like a strict editor, checking the plan for mistakes before anyone starts working. And it has a Memory Manager who keeps a notebook of past successes and failures so the team doesn't make the same mistakes twice. When you ask a question in natural language—like "Show me the total salaries of all basketball players who won more than 10 championships"—this team works together to find the right data, figure out the math, and give you the answer without you needing to write a single line of code.
The researchers behind AgenticData found that this team approach works incredibly well. In their experiments, they tested the system on five different benchmarks, which are like standardized tests for data analysis. The results showed that AgenticData was significantly more accurate than the current best methods. On some difficult tasks, it improved accuracy by nearly 30% compared to other top systems. For example, on a dataset involving real bank data, it was almost 20% more accurate than the next best competitor. The system didn't just guess; it used a clever feedback loop. If the "Validator" found a mistake in the plan, the system didn't just give up. It stored the error in its memory, learned from it, and tried a new, better plan. This process of "trying, failing, learning, and trying again" allowed it to solve complex puzzles that stumped other systems.
One of the most exciting findings is how AgenticData handles the "messy" data. Traditional systems often struggle when they can't find a clear structure in a document. AgenticData, however, uses its agents to break down unstructured text into useful pieces, find connections between different documents, and even link them to structured tables. In tests involving Wikipedia articles and real-world bank records, the system successfully navigated these chaotic data sources to provide precise answers. The researchers also measured the cost and speed of the system. They found that by optimizing how the agents used their "brain power" (specifically, how many times they called upon large language models), AgenticData could achieve high-quality results for about half the cost of other methods.
The paper explicitly argues against the idea that a single, massive AI model can handle all these tasks alone. They found that relying on just one model to plan, execute, and validate everything often leads to errors and "hallucinations" (where the AI makes things up). Instead, their experiments suggest that breaking the job down into a collaborative team with specific roles and a robust memory system is the key to success. They also ruled out the idea that humans need to write code for every new analysis; their system showed it could autonomously generate the necessary steps. While the system is not perfect—it still takes a few minutes to plan complex queries and can sometimes struggle with extremely rare data patterns—the results suggest it is a major step forward. The team has already deployed this system in real-world settings, including banks and power grid companies, where it is currently helping professionals analyze complex data without needing a team of data scientists to write code for every single question.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.