💬 NLP

TraceBack: Multi-Agent Decomposition for Fine-Grained Table Attribution

This paper introduces TraceBack, a multi-agent framework that enhances transparency in table-based question answering by providing fine-grained, cell-level attribution for answers, accompanied by the CITEBench benchmark and the reference-less FairScore metric for systematic evaluation.

Original authors: Tejas Anvekar, Junha Park, Rajat Jha, Devanshu Gupta, Poojah Ganesan, Puneeth Mathur, Vivek Gupta

Published 2026-02-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Tejas Anvekar, Junha Park, Rajat Jha, Devanshu Gupta, Poojah Ganesan, Puneeth Mathur, Vivek Gupta

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you ask a very smart, but sometimes overly confident, librarian (an AI) a question about a giant library of spreadsheets. The librarian gives you a perfect answer. But then you ask, "How do you know that's true? Which specific book or page did you read?"

Often, the librarian just shrugs or points vaguely at the whole shelf. This is a problem. In high-stakes situations (like medical advice or financial reports), you need to know exactly which cells in the table supported the answer.

This paper introduces a new system called TRACEBACK to solve this, along with a new way to test it called CITEBENCH and a new "scorecard" called FAIRSCORE.

Here is the breakdown using simple analogies:

1. The Problem: The "Black Box" Librarian

Current AI systems are like magicians. They pull the right answer out of a hat, but you can't see the trick.

The Issue: If an AI says, "Solar power is the most efficient," it might be right, but it might have guessed. We need it to point to the exact row and column in the table that says "Solar: 30-50% efficiency."
The Old Way: Previous tools were like a blurry camera. They could tell you which row had the answer, but they couldn't tell you which specific word in that row was the key. They also missed the "middle steps" (like how the AI calculated a duration by looking at a start time and an end time).

2. The Solution: TRACEBACK (The Detective Team)

The authors built TRACEBACK, which isn't just one AI, but a team of specialized AI agents working together like a detective squad. Instead of guessing the answer in one big leap, they break the case down:

Agent 1 (The Filter): "Okay, we are looking for 'Solar Power.' Let's throw away every row in the table that isn't about Solar." (Pruning the table).
Agent 2 (The Decomposer): "The question is complex. Let's break it into smaller clues. First, find the cost. Then, find the scalability. Finally, find the efficiency."
Agent 3 (The Evidence Hunter): "Now, for the 'Cost' clue, point to the exact cell that says '$30-50'."
Agent 4 (The Connector): "Okay, we have the cost cell, the scalability cell, and the efficiency cell. Let's stitch them together to prove the final answer."

The Result: TRACEBACK doesn't just give you the answer; it gives you the receipt. It shows you exactly which cells were used for every step of the reasoning, even the hidden steps.

3. The New Test: CITEBENCH (The Gold Standard Exam)

To see if TRACEBACK actually works, the authors needed a test. They realized existing tests were flawed:

The Flaw: Old tests were like grading a student's essay by only checking if the final sentence was right, ignoring the math they did to get there. Also, some old tests had "noisy" answers (marking irrelevant cells as important).
The Fix: They created CITEBENCH. Imagine a teacher manually grading 1,500 exam papers. For every answer, the teacher draws a red circle around the exact cells in the table that prove the answer is right. This creates a "Gold Standard" to measure how good the AI is at finding the truth.

4. The Scorecard: FAIRSCORE (The "No-Answer-Key" Grader)

Here is the tricky part: Manually grading 1,500 papers is expensive and slow. What if you want to test the AI on 100,000 questions? You can't hire a human for that.

Enter FAIRSCORE.

The Analogy: Imagine you don't have the answer key, but you have the student's answer and the textbook.
How it works: FAIRSCORE takes the AI's answer and breaks it into tiny "atomic facts" (e.g., "Solar costs $30"). It then takes the cells the AI pointed to and turns them into facts too ("The table says Solar costs $30").
The Match: It compares the two lists.
- If the AI's facts match the cells it pointed to, it gets points for Precision (it didn't lie).
- If the AI's facts cover everything in the answer, it gets points for Recall (it didn't miss anything).
Why it's cool: It can grade the AI's work without needing a human to check the answer key first. It's like a self-correcting homework helper.

5. The Results

When they ran the tests:

TRACEBACK crushed the competition. It was much better at finding the exact right cells than previous methods.
FAIRSCORE proved to be a reliable judge. Even without human help, it correctly identified which AI was doing a better job.

Summary

Think of this paper as building a transparent kitchen.

Old AI: Serves you a delicious stew but won't tell you what's in it.
TRACEBACK: Serves you the stew and hands you a detailed receipt showing exactly which carrots, potatoes, and spices were used, and how they were chopped.
CITEBENCH: A master chef who checks the receipt to make sure it's accurate.
FAIRSCORE: A smart robot that can taste the stew and check the receipt to see if they match, even if the master chef isn't in the room.

This makes AI trustworthy again, especially when you need to know why it gave you an answer, not just what the answer is.

1. Problem Statement

While Large Language Models (LLMs) have improved Question Answering (QA) over structured tables, they frequently suffer from hallucinations and opaque reasoning. Existing systems lack fine-grained attribution, meaning they cannot reliably link specific parts of an answer (or intermediate reasoning steps) to the exact cells in a table that support them.

Current limitations include:

Coarse Granularity: Prior work (e.g., MATSA) operates at the row-column level, failing to distinguish which specific cell supports a specific phrase in an answer.
Missing Intermediate Evidence: Systems often attribute only the final answer, ignoring the implicit cells used for intermediate calculations (e.g., start/end times used to compute a duration).
Evaluation Bottlenecks: Existing benchmarks (like TabCite) suffer from noisy ground truth labels and lack phrase-to-cell alignment, making rigorous evaluation expensive and subjective.

2. Methodology

The paper introduces three core components to address these gaps: a new benchmark, a multi-agent framework, and a reference-less evaluation metric.

A. CITEBENCH: A New Benchmark

To enable systematic evaluation, the authors constructed CITEBENCH, a dataset of 1,500 manually annotated examples derived from ToTTo, FetaQA, and AITQA.

Granularity: Unlike previous benchmarks, CITEBENCH provides phrase-to-cell alignment. Every semantic unit in an answer is explicitly linked to the supporting table cell(s).
Scope: It covers both final answers and intermediate reasoning steps (e.g., filtering conditions, aggregation inputs).
Quality Control: The authors identified significant noise in the original datasets (e.g., 55.3% error rate in FetaQA attributions) and curated a "Gold Set" with high inter-annotator agreement ( $\kappa = 0.72$ ).

B. TRACEBACK: Multi-Agent Framework

TRACEBACK is a modular, post-hoc framework designed to trace answers back to precise cell-level evidence. It utilizes a pipeline of specialized LLM agents:

Column Relevance Identification: Identifies necessary columns (explicit and implicit) required to derive the answer, even if column names aren't in the query (e.g., identifying "Start Time" and "End Time" to answer a "Duration" question).
Evidence Span Extractor: Filters rows based on schema-aware conditions (e.g., WHERE Cost <= 50), reducing the table to relevant rows.
Query Decomposition: Breaks the original question into semantically coherent sub-questions corresponding to intermediate reasoning steps. This exposes implicit dependencies often missed by holistic models.
Sub-Query Attribution: Grounds each sub-question to specific cells in the pruned table.
Final Attribution: Consolidates intermediate attributions into a complete phrase-to-cell mapping, aligning cell coordinates with answer spans.

C. FAIRSCORE: Reference-Less Metric

To overcome the cost of manual annotation for large-scale evaluation, the authors propose FAIRSCORE.

Mechanism: It converts predicted cells and the reference answer into atomic facts (declarative statements).
Alignment: An LLM compares these fact sets to determine if the predicted cells support the answer facts.
Metrics: It calculates Precision (how many predicted cells are actually needed) and Recall (how much of the answer is covered by the predicted cells) without requiring gold-standard cell labels.

3. Key Contributions

CITEBENCH: The first benchmark offering phrase-to-cell annotations for both final answers and intermediate reasoning steps across diverse table domains (Wikipedia, financial reports).
TRACEBACK: A novel multi-agent framework that combines schema-aware pruning, query decomposition, and fine-grained grounding to achieve high-fidelity cell attribution.
FAIRSCORE: A scalable, reference-less evaluation metric that correlates strongly with human judgments, enabling evaluation on unlabeled datasets.
Empirical Validation: Extensive experiments demonstrating state-of-the-art performance and the effectiveness of the proposed metric.

4. Experimental Results

The authors evaluated TRACEBACK against strong baselines (e.g., Few-shot ICL, Post-hoc Retrieval, INSEQ, GENERATEPROGRAMS) on ToTTo, FetaQA, and AITQA.

Performance Gains: TRACEBACK substantially outperforms baselines across all granularities (Row, Column, Cell).
- Cell-Level Precision: On FetaQA, TRACEBACK achieved 89.81% precision compared to the next best baseline (INSEQ) at 56.50%.
- Recall: It maintains high recall (e.g., 97.12% on AITQA), demonstrating that schema-aware filtering avoids the over-retrieval common in retrieval-based baselines.
Ablation Studies:
- Removing Query Decomposition caused a significant drop in precision (e.g., from 89.81% to 65.40% on FetaQA), proving its necessity for capturing intermediate evidence.
- Removing Table Pruning reduced recall and efficiency.
FAIRSCORE Validation:
- FAIRSCORE scores closely track human judgments, preserving the relative ranking of methods.
- While it slightly underestimates absolute precision (by ~16 points on average), it effectively distinguishes between strong and weak models, validating its use as a scalable proxy for gold-label evaluation.

5. Significance and Future Work

Significance:

Trust & Transparency: TRACEBACK provides the necessary transparency for high-stakes table QA by verifying not just what the answer is, but exactly which data points support it, including implicit reasoning steps.
Scalable Evaluation: FAIRSCORE solves the bottleneck of manual cell-level annotation, allowing for the evaluation of attribution quality on massive, unlabeled datasets.
Methodological Shift: The paper shifts the paradigm from "answer generation" to "evidence-grounded reasoning," highlighting that decomposing queries is essential for accurate attribution in structured data.

Limitations & Future Directions:

Scope: Currently limited to single-table settings; multi-table joins and hierarchical schemas are not yet supported.
Dependency: Relies heavily on LLM performance, making it sensitive to prompt design and model changes.
Generalization: CITEBENCH is currently English-only; future work aims to extend to multilingual and multimodal settings.

In conclusion, TRACEBACK represents a significant step forward in making structured data QA interpretable and verifiable, providing both a robust framework for attribution and a practical tool for its evaluation.