💬 NLP

ODUTQA-MDC: A Task for Open-Domain Underspecified Tabular QA with Multi-turn Dialogue-based Clarification

This paper introduces the ODUTQA-MDC task and benchmark, featuring a large-scale dataset and a multi-agent framework (MAIC-TQA) designed to address underspecified open-domain tabular questions through multi-turn dialogue-based clarification.

Original authors: Zhensheng Wang, ZhanTeng Lin, Wenmian Yang, Kun Zhou, Yiquan Zhang, Weijia Jia

Published 2026-04-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhensheng Wang, ZhanTeng Lin, Wenmian Yang, Kun Zhou, Yiquan Zhang, Weijia Jia

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to find a specific house in a massive, chaotic library of real estate records. You ask the librarian (an AI), "How is the environment of that new community?"

The librarian looks at you, confused. There are thousands of "new communities." Which one? In which city? Which district? Because your question was vague, the librarian might guess the wrong one, or worse, give you a "No results found" error.

This paper introduces a solution to that exact problem. It's like upgrading the librarian from a rigid robot that just follows orders to a smart, chatty detective who knows how to ask the right follow-up questions to get the answer you actually need.

Here is the breakdown of their work, explained simply:

1. The Problem: The "Vague Question" Trap

Current AI models are great at answering questions when the instructions are perfect. But in the real world, people are messy. We make typos, forget details, or use slang.

The Old Way: If you ask, "What's the greenery rate?" without saying which city, the AI might pick a random city, get the wrong answer, and confidently tell you it's correct. Or it might just crash.
The New Challenge: The authors call this "Underspecified Tabular QA." It's the art of answering questions from huge databases when the user hasn't given enough info.

2. The Solution: ODUTQA-MDC (The New Playground)

To teach AI how to handle these vague questions, the researchers built a new "training gym" called ODUTQA-MDC.

The Dataset: They created a massive library of 209 tables (like spreadsheets for real estate, land auctions, and finance) with over 25,000 questions.
The Twist: They intentionally made the questions vague. Some were missing the city name, some were missing the project name, and some were just too general.
The Simulator: Instead of hiring thousands of humans to chat with the AI (which is expensive and slow), they built a "User Simulator." This is a digital actor that pretends to be a confused human. If the AI asks, "Which city?" the simulator replies, "Oh, sorry, I meant Shenzhen."

3. The Hero: MAIC-TQA (The Detective Team)

The researchers didn't just build the gym; they built a new AI system called MAIC-TQA to train in it. Think of this system not as one brain, but as a team of four specialized detectives working together:

The Listener (SLU Module): This agent listens to your question and tries to figure out what you really want. If you say, "How's the environment?", it realizes you are asking about "greenery" or "facilities" but isn't sure which.
The Scope Checker (Scope Validator): This agent checks if you gave enough location details. If you forgot the city, it raises a flag: "Wait, we need a city to find the right table!"
The Librarian (Table Retrieval): Once the clues are gathered, this agent scans the massive database to find the exact spreadsheet you need.
The Calculator (SQL Generator): This agent writes the code (SQL) to pull the specific numbers from that spreadsheet and gives you the final answer.

The Magic Loop: If the Calculator tries to run the code and gets an error (like "No data found"), the whole team stops. They don't give up. They go back to the Listener, say, "Hey, the user's question about 'Binjiang Garden' didn't match any records. Let's ask them to clarify the name," and start the conversation again.

4. The Results: Why It Matters

The researchers tested their new "Detective Team" against other top AI models.

Without Clarification: Other models tried to guess and failed often, giving wrong answers or crashing.
With Clarification (MAIC-TQA): By having that "chat" to clear up confusion, their system got the right answer much more often (sometimes doubling the success rate).

The Big Picture Analogy

Imagine you are ordering food at a restaurant.

Old AI: You say, "I want the spicy chicken." The chef guesses which spicy chicken you mean, makes it, and you hate it because you wanted the Thai spicy chicken, not the Sichuan one.
New AI (MAIC-TQA): You say, "I want the spicy chicken." The waiter (the AI) says, "Sure! Do you mean the Thai one with lemongrass, or the Sichuan one with numbing peppers?" You say, "Thai!" The waiter confirms, "Got it, Thai spicy chicken coming up."

In summary: This paper teaches AI to stop guessing when humans are vague and start having a conversation to get the facts straight. It's a huge step toward making AI assistants that are actually helpful in the messy, real world.

1. Problem Definition

The paper addresses a critical gap in Open-Domain Tabular Question Answering (Tabular QA). While Large Language Models (LLMs) have advanced text-to-SQL capabilities, they struggle with underspecified queries in real-world open-domain scenarios.

The Challenge: Real-world user queries often contain ambiguities (spelling errors, missing context, vague expressions) that prevent the generation of correct SQL queries. Unlike closed-domain settings where the table is known, open-domain systems must first retrieve the correct table from a large database.
The Gap: Existing methods typically fail to handle queries where the ambiguity requires multi-turn clarification to resolve. Current datasets either lack underspecification or rely on static, predetermined dialogues that do not simulate the dynamic nature of human interaction.
Specific Ambiguities: The paper categorizes underspecification based on SQL clauses:
- SELECT (Intent): Unclear user intent (e.g., "How is the environment?" vs. "What is the greenery rate?").
- FROM (Scope): Missing or incorrect table scope information (e.g., missing city/district).
- WHERE (Condition): Invalid or missing entity names (e.g., typos or non-existent project names).

2. Methodology

The authors propose a comprehensive solution comprising a new task definition, a large-scale dataset, a fine-grained evaluation framework, and a multi-agent system.

A. ODUTQA-MDC Task & Dataset

ODUTQA Dataset: The first large-scale open-domain underspecified Tabular QA dataset.
- Scale: 209 structured tables and 25,105 QA pairs derived from real estate, property, and land auction domains.
- Construction: Based on 90 seed templates from the RETQA dataset. The authors systematically injected underspecification by manipulating templates (restructuring) and values (modification) to create SELECT, FROM, and WHERE ambiguities, as well as Mixed types.
- Annotation: Includes fine-grained labels for Intent, Scope, and Condition underspecification using a triple-based format [content, type, error_type] (e.g., ["", "District", "Missing"]).
Dynamic Clarification Interface: An automated user simulator that interacts with the QA system.
- It detects underspecifications and triggers clarification questions.
- It offers Fixed Mode (standardized responses for reproducibility) and Dynamic Mode (LLM-paraphrased colloquial responses to test robustness).
- It enforces a "detection-clarification-redetection" closed loop.

B. MAIC-TQA Framework

The authors propose MAIC-TQA (Multi-Agent Interactive Clarification for Tabular QA), an LLM-powered multi-agent system designed to handle the ODUTQA-MDC task. It consists of four collaborative agents:

SLU Module (Spoken Language Understanding): Uses a BERT-based classifier to extract user intent and slots. It detects Intent Underspecification and triggers clarification if the intent is "Unknown."
Scope Validator Agent (SV Agent): Validates slot values against the database schema. It identifies Scope Underspecification (Missing, Error, or Unmatch types) and prompts the user for clarification on table scope (e.g., city/district).
Table Retrieval Agent (TR Agent): Generates a summary of the target table based on the original query and user clarifications. It uses BM25 for similarity search to retrieve the correct table caption from the database.
SQL Generation and Validation Agent (SGV Agent): Generates SQL queries using in-context learning (5 gold examples). It executes the SQL and validates the result. If the result is empty or incomplete, it identifies Condition Underspecification and initiates a final round of clarification.

3. Key Contributions

New Task & Benchmark: Introduction of ODUTQA-MDC, the first task specifically targeting open-domain underspecified Tabular QA with multi-turn clarification.
ODUTQA Dataset: A large-scale dataset (25k+ pairs) with fine-grained annotations for three types of underspecification (Intent, Scope, Condition) and a dynamic clarification interface.
MAIC-TQA Framework: A novel multi-agent architecture that integrates adaptive underspecification detection with iterative clarification, significantly improving adaptability in complex scenarios.
Comprehensive Evaluation: The first benchmark to assess the entire workflow: "Information Detection $\rightarrow$ Multi-turn Clarification $\rightarrow$ QA Reasoning."

4. Experimental Results

The authors evaluated MAIC-TQA against the state-of-the-art baseline SLUTQA (which lacks clarification mechanisms) across various LLM backbones (Qwen3, Kimi K2, GLM 4, Doubao).

Underspecification Detection:
- BERT-based approaches achieved high accuracy (>99%) for intent detection.
- For scope and condition detection, models like Doubao Seed 1.6 and Kimi K2 showed strong performance, with F1 scores reaching up to 89.46% for scope underspecification in dynamic settings.
Table Retrieval:
- MAIC-TQA significantly outperformed SLUTQA. For example, on Qwen3 32B, Table Retrieval F1 improved from 60.07% (SLUTQA) to 92.89% (MAIC-TQA Fixed).
- Reason: Proactive identification of missing scope information (e.g., city/district) allows for precise table retrieval.
SQL Generation & Execution:
- MAIC-TQA demonstrated superior Execution Correctness Rate (ECR) and Pass@1.
- On Doubao Seed 1.6, ECR improved by 12.29% (Fixed) and 19.04% (Dynamic) over the baseline.
- On Qwen3 32B, Pass@1 improved by 33.24% (Fixed) and 33.69% (Dynamic).
Ablation Study: Removing clarification mechanisms for Scope (FROM) and Condition (WHERE) caused the most significant performance drops (e.g., Pass@1 dropped by ~28% when WHERE clarification was removed), confirming that these are the primary bottlenecks in underspecified QA.

5. Significance

Bridging the Reality Gap: The work moves Tabular QA from idealized, well-specified queries to realistic, ambiguous user interactions, which is crucial for practical deployment.
Methodological Advancement: It demonstrates that multi-agent collaboration combined with iterative clarification is a superior strategy for handling ambiguity compared to single-pass text-to-SQL approaches.
Resource for Research: By releasing the dataset, labeling scheme, and code, the authors provide a standardized benchmark to drive future research in conversational, ambiguity-aware database interaction.
Scalability: The use of a simulated dynamic interface allows for reproducible, large-scale evaluation of models' ability to resolve logical ambiguities without the cost and inconsistency of human-in-the-loop testing.