← Latest papers
🤖 AI

LeafData: An Agentic System for Data Migration

LeafData is an agentic system that streamlines data migration by converting user intent into validated, executable JSON configurations through a chatbot-driven interface, thereby eliminating the need for manual coding and domain expertise.

Original authors: Sadanand Katukuri, Rajasekhar Bada, Navya Induri, Rohit Gandham, Lynette Pinto, Joses Selvan, Abishek Krishnamoorthy, Joseph Rozario, Pu Tian, Pavan Poudel, Yalong Wu

Published 2026-07-27
📖 7 min read🧠 Deep dive

Original authors: Sadanand Katukuri, Rajasekhar Bada, Navya Induri, Rohit Gandham, Lynette Pinto, Joses Selvan, Abishek Krishnamoorthy, Joseph Rozario, Pu Tian, Pavan Poudel, Yalong Wu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a massive, intricate Lego castle, but the instructions are written in a secret code that only a few experts can read. This is the current reality of moving data from one computer system to another. In the world of data science, "data migration" is simply the act of packing up information from one place (like an old filing cabinet) and moving it to a new home (like a shiny new digital cloud). To do this, engineers usually have to write complex "recipes" called pipelines. These pipelines tell the computer exactly how to grab the data, clean it up, and drop it off in the right spot. Traditionally, writing these recipes requires a special language called JSON, which is like a strict, unforgiving grammar that demands perfect punctuation and specific rules. If you miss a comma or get a number wrong, the whole thing crashes. This creates a huge barrier: if you aren't a coding wizard, you can't easily move your data, even if you know exactly what you want to happen.

Enter LeafData, a new system designed to be the friendly translator between your everyday thoughts and that strict computer code. Think of LeafData not as a robot that just follows orders, but as a helpful, super-smart tour guide. Instead of forcing you to learn the secret code, you just tell the guide, "I want to move my customer list from this old database to that new spreadsheet." The guide then asks you a series of simple questions, one by one, like a detective solving a mystery. It checks your answers against a strict rulebook to make sure you haven't made any mistakes before it writes the code. Once it has all the pieces, it automatically builds the perfect "recipe" (the JSON configuration) and hands it to a master builder (an orchestration platform) who actually does the heavy lifting of moving the data. The result is that anyone, even a curious teenager with a great idea, can move complex data without ever needing to learn the secret code.

The Paper's Core Discovery

The paper introduces LeafData, an "agentic system" (which is just a fancy way of saying a smart, autonomous helper) that turns natural language user requests into validated, executable data migration pipelines. The authors found that by combining a chatbot interface with a strict validation engine, they could eliminate the need for users to manually write complex configuration files.

Here is how the magic happens, step-by-step:

1. The Chatbot Guide (The Frontend)
Imagine you are talking to a very patient librarian who knows exactly what you need to build a bridge. You start by saying, "I want to move data from my MySQL database to a file on AWS S3." The chatbot doesn't just say "Okay" and hope for the best. Instead, it enters a "state tracking" mode. It knows you've told it the source and the destination, but it knows you haven't told it the host name or the port number yet. It asks for these details one by one.

  • The Safety Net: If you type "Port: abc" (which is nonsense because a port must be a number), the chatbot immediately stops you. It says, "Port must be numeric. Please try again." It uses a strict rulebook (called an Apache Avro schema) to check every single answer before moving on. This ensures that by the time the system is done talking to you, your instructions are 100% correct.
  • The "Time Travel" Feature: If you change your mind, you don't have to start over. You can use a special command like @change to say, "Actually, I meant to use a different database name," and the system updates its internal notes without losing any of the other correct information you already gave.

2. The Architect (The Backend)
Once the chatbot has collected all the correct, validated information, it hands the notes to the backend service. This part of the system is like an architect who takes your simple list of requirements and draws up the blueprints.

  • The Blueprint: The system generates specific JSON files. These aren't just random text files; they are structured "artifacts" that define exactly how to connect to the source, how to connect to the destination, and the step-by-step plan (called a DAG, or Directed Acyclic Graph) for moving the data.
  • The Translator: The system doesn't care if you are moving data from a database, a spreadsheet, or a website API. It has a "connector abstraction layer," which is like a universal adapter. Whether you are plugging in a USB drive or a fiber optic cable, the adapter makes sure the power flows the same way. This means the system can handle MySQL, Oracle, MongoDB, SFTP files, and REST APIs all using the same underlying logic.

3. The Builder (Orchestration)
Finally, these JSON blueprints are fed into a master builder called Apache Airflow. Airflow reads the files and builds the actual pipeline. It creates a visual graph (like a flowchart) showing the tasks: "First, grab the data from the source. Second, save it to a temporary spot. Third, load it into the destination."

  • The Result: The paper demonstrates this with real examples. In one case, they moved medical records from an AWS S3 bucket (a cloud storage folder) into a MySQL database. The system automatically created a task to download the file, a task to clean it up, and a task to upload it. The result was a successful transfer with a clear log of what happened.
  • Complex Data: They also showed it working with "messy" data, like documents from MongoDB (which have nested, irregular structures) or JSON from a REST API. The system automatically flattened these complex structures into neat, organized rows for the database, proving it can handle different types of data without human intervention.

What LeafData Does Not Do (Yet)

It is important to understand the boundaries of this system. The paper explicitly states that LeafData currently supports linear pipelines. This means it is great for moving data from Point A to Point B in a straight line. It does not yet handle complex workflows where the path splits (branching) or where the system has to make decisions based on the data (conditional logic). For example, it can't say, "If the data is big, go to the cloud; if it's small, go to the local server." That is a job for future versions of the system.

The Verdict

The authors are confident that this approach works. They didn't just suggest it might work; they built a working prototype and tested it with real data migrations. They showed that by using a chatbot to guide the user and a strict validator to check the answers, they could generate error-free configuration files that successfully moved data across different systems.

The key takeaway is that LeafData suggests a shift in how we interact with technology: instead of forcing humans to learn the machine's strict language, we can teach the machine to understand our natural language, provided we have a strict "grammar police" (the validation layer) to make sure we are precise. This makes the powerful world of data migration accessible to non-experts, reducing the time it takes to set up pipelines and removing the fear of making a typo that breaks the whole system.

Drowning in papers in your field?

Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.

Try Digest →