Agentic DAG-Orchestrated Planner Framework for Multi-Modal, Multi-Hop Question Answering in Hybrid Data Lakes

The paper introduces A.DOT, an agentic framework that orchestrates multi-modal, multi-hop question answering over hybrid data lakes by compiling natural language queries into directed acyclic graph execution plans, thereby significantly improving accuracy, latency, and transparency compared to traditional retrieval methods.

The Gist

Imagine you are a detective trying to solve a complex mystery. You have two types of evidence:

1. The Ledger: A neat, organized spreadsheet with numbers, dates, and names (Structured Data).
2. The Case Files: A massive pile of unorganized letters, contracts, and handwritten notes (Unstructured Data).

Your boss asks a tricky question: "Who are the clients from Texas who paid late, and what did their contract say about penalties?"

To answer this, you can't just look at the spreadsheet (you won't find the contract details there) and you can't just read the letters (you won't know which ones belong to Texas clients without the spreadsheet). You need to jump back and forth between the two.

The Problem with Old Systems:
Most current AI systems are like a frantic intern who, when given this question, grabs every single piece of paper from the file cabinet and every single row from the spreadsheet, dumps them on your desk, and says, "Here, you figure it out!"

• The Result: It's slow, overwhelming, and often misses the point because the AI gets lost in the noise. It's also risky because it might accidentally show you confidential info you didn't need to see.

The Solution: A.DOT (The Master Architect)
The paper introduces a new system called A.DOT (Agentic DAG-Orchestrated Transformer). Think of A.DOT not as a frantic intern, but as a Master Architect who draws a precise blueprint before building anything.

Here is how A.DOT works, using simple analogies:

1. The Blueprint (DAG Planning)

Instead of guessing, A.DOT first pauses to draw a flowchart (called a Directed Acyclic Graph, or DAG).

• Step 1: "Go to the Spreadsheet, find all clients in Texas."
• Step 2: "Take those names and go to the File Cabinet to find their contracts."
• Step 3: "Read the contracts to find the penalty clauses."
• Step 4: "Calculate the average penalty."

This blueprint ensures the AI knows exactly where to go next. It doesn't waste time reading irrelevant files.

2. The Safety Inspectors (Validation & DataOps)

Before the AI actually starts fetching data, two safety inspectors check the blueprint:

• The Structural Inspector: Checks if the spreadsheet actually has a "Texas" column. If the blueprint says "Go to the 'City' column" but the spreadsheet only has "State," the inspector stops the plan and says, "Fix this first!"
• The Semantic Inspector: Checks if the plan makes sense. "Wait, you're asking for 'penalties' in a contract that only talks about 'delivery dates.' That doesn't match."

If the plan is broken, a Repair Crew (DataOps) steps in. They don't just give up; they try to fix the blueprint on the fly or ask for a new one, ensuring the AI never crashes or gives a wrong answer.

3. The Relay Race (Parallel Execution)

Once the blueprint is approved, the work begins.

• Smart Teamwork: If two steps don't depend on each other (e.g., checking "Texas" clients and checking "California" clients), A.DOT sends two different agents to do them at the same time. This is like a relay race where runners start as soon as they are ready, rather than waiting for everyone to line up.
• The Handoff: When one agent finds a list of names, they don't hand over the entire spreadsheet to the next agent. They just pass a small sticky note with the specific names needed. This keeps the process fast and prevents the "data overload" problem.

4. The Memory Bank (Caching)

If you ask the same question again (or ask it in a slightly different way, like "Show me Texas clients" vs. "Who is from the Lone Star State?"), A.DOT checks its Memory Bank.

• If it sees it solved this exact puzzle yesterday, it grabs the old blueprint and reuses it. No need to draw a new one! This makes the system incredibly fast for repeated questions.

5. The Paper Trail (Lineage & Trust)

Every time A.DOT fetches a piece of data, it writes down exactly where it came from.

• "I found the penalty clause in Contract #402, Page 3."
• "I found the Texas address in Row 15 of the Invoice Table."
  This creates a verifiable trail. If a human manager asks, "How did you get that number?", A.DOT can point to the exact source. This builds trust, which is crucial for big companies.

Why is this a Big Deal?

The paper tested A.DOT against other smart AI systems (like the "frantic intern" or the "ReAct" agent) using a difficult test called HybridQA.

• The Result: A.DOT got the right answer 14.8% more often and provided a more complete answer 10.7% more often than the competition.

In Summary:
A.DOT is like a highly organized, super-smart project manager. It doesn't just "search and guess." It plans, checks its work, fixes its own mistakes, works in parallel, and keeps a detailed log of everything it does. This makes it perfect for complex business questions where accuracy and trust are everything.

Technical Summary

1. Problem Statement

Enterprises increasingly utilize Hybrid Data Lakes, which combine structured data (relational tables) and unstructured data (documents, contracts, logs). While users expect to query these heterogeneous sources using Natural Language (NL), current solutions face significant limitations:

• Inefficiency & Leakage: Existing Retrieval-Augmented Generation (RAG) systems often rely on "brute-force" retrieval from all stores, leading to high computational overhead and potential data leakage by exposing more information than necessary.
• Lack of Multi-Hop Reasoning: Current systems struggle with multi-hop reasoning, where a query requires decomposing into successive steps that traverse back and forth between structured and unstructured sources (e.g., finding a specific ID in a table, then retrieving a document linked to that ID).
• Sequential Latency: Agentic frameworks like ReAct execute sub-tasks sequentially, increasing latency.
• Lack of Provenance: Existing approaches often fail to provide explicit evidence trails or data lineage, which is critical for enterprise trust and compliance.

2. Methodology: The A.DOT Framework

The authors propose A.DOT (Agentic DAG-Orchestrated Transformer), a framework that compiles NL queries into Directed Acyclic Graph (DAG) execution plans. The system is designed to handle hybrid data ($D = \{D_S, D_U\}$) through the following components:

A. Architecture Components

1. Plan Caching: Accelerates processing by reusing validated DAG plans for semantically equivalent queries (using exact, template, and semantic matching).
2. Plan Generator: A single LLM pass converts the NL query into a DAG where nodes represent atomic sub-queries mapped to specific data sources (SQL for structured, Vector for unstructured).
3. Plan Validator: Performs dual validation before execution:
   • Structural: Checks schema compliance, variable hygiene, and acyclicity.
   • Semantic: Ensures intent preservation and execution feasibility (e.g., valid join keys).
4. DataOps System: A feedback loop for diagnosis and remediation. If validation or execution fails, it triggers:
   • Diagnoser: Identifies root causes (e.g., tool mismatch).
   • Recommender: Suggests recovery actions.
   • Fixer: Applies conservative local edits.
   • Replanner: Rewrites the DAG for structural issues.
5. Plan Executor:
   • Executes independent nodes in parallel.
   • Uses a variable-binding mechanism to pass only minimal required data (e.g., IDs) between nodes, preventing context overflow and reducing data leakage.
   • Supports streaming of intermediate results.
   • Maintains lineage logs for every step (inputs, outputs, source provenance).

B. Execution Flow

1. Input: User NL Query.
2. Planning: Query is converted to a DAG plan.
3. Validation: Structural and semantic checks are performed. If errors occur, the DataOps system intervenes.
4. Execution: The engine traverses the DAG topologically. Parallel nodes run concurrently; dependent nodes receive filtered intermediate variables.
5. Synthesis: A final response module aggregates results into a coherent, human-readable answer with an evidence trail.

3. Key Contributions

1. Unified Agentic Framework: A novel system combining DAG-based planning, structural/semantic validation, and a dual-stage DataOps system for self-healing.
2. Variable-Binding Mechanism: A method to selectively pass data elements between structured and unstructured sub-queries, improving scalability and reducing data leakage.
3. Parallel Execution Strategy: Exploits DAG independence to reduce end-to-end latency while exposing intermediate results for responsiveness.
4. Plan-Driven Provenance: A model that records evidence and data lineage at every stage, enabling auditability and verification.
5. Empirical Superiority: Demonstrated significant gains over strong baselines on the HybridQA benchmark.

4. Experimental Results

The system was evaluated on the HybridQA dataset (3,466 dev samples), adapted to a hybrid SQL + Vector (Milvus) architecture.

• Baselines Compared: Standard RAG, ReAct (iterative reasoning), and LLM Compiler (DAG-based).
• Performance Metrics:
  • Answer Correctness: A.DOT achieved 71.0%, outperforming the next best (Standard RAG at 56.2%) by 14.8% absolute gain.
  • Answer Completeness: A.DOT achieved 73.0%, outperforming Standard RAG (62.3%) by 10.7%.
• Ablation Study:
  • Removing the DataOps System caused the sharpest performance drop (Correctness fell to 60.0%), highlighting its critical role in recovering from execution errors.
  • Removing both the Validator and DataOps System resulted in a slight performance increase (67.9%) compared to having only the Validator, suggesting that without a repair mechanism, the system sometimes succeeds by "blindly" executing despite errors, whereas the Validator alone halts execution without fixing it.
• Error Analysis: The system effectively handles intermediate subquery failures and incorrect data source assignments, though challenges remain with intrinsic commonsense knowledge.

5. Significance and Future Work

• Enterprise Readiness: A.DOT is currently being evaluated for deployment as the core retrieval module in IBM Watsonx.data Premium. It addresses critical enterprise needs: accuracy, low latency, data security (minimal leakage), and strict compliance (lineage tracking).
• Scalability: The framework moves beyond simple retrieval to complex, multi-modal reasoning, bridging the gap between relational databases and vector stores.
• Future Directions: The authors plan to extend A.DOT to support multi-turn conversational settings and integrate additional modalities like graph databases and image-based knowledge.

In summary, A.DOT represents a significant advancement in enterprise AI, shifting from brute-force retrieval to a principled, plan-driven, and self-correcting architecture capable of complex reasoning across hybrid data environments.