💬 NLP

Towards Detecting Inconsistencies in End-to-end Generated TODs

This paper proposes a novel method for automatically detecting inconsistencies in end-to-end generated Task-Oriented Dialogues by modeling the dialogue as a Constraint Satisfaction Problem (CSP) to identify hallucinations and suggest minimal corrections for adherence to domain knowledge.

Original authors: Tiziano Labruna, Giovanni Bonetta, Bernardo Magnini

Published 2026-07-13

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Tiziano Labruna, Giovanni Bonetta, Bernardo Magnini

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are playing a high-stakes game of "Guess the Restaurant" with a super-smart, but slightly scatterbrained, AI chef. The chef has a secret menu (a Knowledge Base) listing exactly which restaurants exist, what food they serve, and how much they cost. Your job is to order a meal. The chef's job is to listen to your request and tell you what's available.

In the old days, the chef was a robot with a strict rulebook: "If the user asks for Spanish food, check the list. If the list says '2', say '2'." But now, we've upgraded the chef to a Generative AI (a Large Language Model). This new chef is incredibly creative and can chat like a human, but it has a dangerous habit: it sometimes "hallucinates." It might confidently tell you there are three Spanish restaurants when the secret menu only lists two, or it might suggest a Lebanese restaurant when you've been asking for Spanish food the whole time.

This paper is about building a "Fact-Checker" to catch these lies before they ruin your dinner.

The Problem: The Creative Chef's Slip-ups

The authors found that even the best AI chefs get it wrong surprisingly often. In one study, they showed that top-tier open-source AI models get the facts wrong in up to 59% of the conversations they generate. If the AI says a restaurant is cheap when it's actually expensive, or invents a place that doesn't exist, the whole conversation fails.

The paper argues against the idea that we can just trust the AI to "know" the facts. Instead, they propose that we need to treat the conversation like a logic puzzle.

The Solution: The "Logic Puzzle" Detective

The authors' main idea is to view a conversation not just as chat, but as a Constraint Satisfaction Problem (CSP).

Think of a CSP like a Sudoku game.

The Variables: These are the specific details in the conversation, like "How many restaurants?" or "What type of food?"
The Constraints: These are the rules. For example, "The number of restaurants mentioned must match the number on the secret menu," or "The food type must stay the same throughout the chat."
The Solver: This is a specialized computer program (a CSP solver) that acts like a super-fast detective. It tries to fill in the blanks of the conversation to see if they fit the rules.

Here is how their "detective pipeline" works, step-by-step:

Spot the Variables: First, the system scans the chat and highlights the important bits (like "Spanish" or "three").
Set the Rules: It writes down the rules based on the secret menu and the logic of conversation (e.g., "If you said 'Spanish' earlier, you can't switch to 'Lebanese' later without a good reason").
Run the Solver: The detective tries to find a valid solution. It asks, "Is there any way to fill in these blanks that matches the secret menu and the conversation flow?"
The Verdict: If the AI's chat matches a valid solution, it's consistent. If the AI's chat doesn't match any valid solution, the detective flags it as inconsistent and even suggests the smallest change needed to fix it (like changing "three" to "two").

What They Found (The Numbers)

The authors tested this "Logic Puzzle" detective on a dataset of 108 dialogue pairs. They compared their method against a few other approaches:

Random Guessing: If you just guessed "consistent" or "inconsistent" like flipping a coin, you'd get it right 50.0% of the time.
Perfect Human Annotations: If they used perfect, pre-labeled data to feed the detective, it got it right 91.6% of the time. This proves the logic puzzle idea works really well when the data is clean.
The Real-World Test (GPT-4o): When they used a modern AI (GPT-4o) to automatically find the variables and then ran the detective, the system achieved an accuracy of 75.9%.

This suggests that while the AI isn't perfect at spotting its own mistakes yet, the CSP method is a very strong tool for catching them.

Testing the AI's "Memory"

The authors also ran a different experiment to see how well AI models could follow the rules when asked to rewrite a conversation. They took a chat, erased the specific details (like restaurant names), and asked the AI to fill them back in using the secret menu.

The results were a bit humbling for the AI:

Even the smartest models (GPT-4o and GPT-o1) only got the entire conversation right 14% of the time (Global Consistency Accuracy).
However, they got individual details right about 41-42% of the time (Variable Consistency Accuracy).

This suggests that while AI is getting better at following rules, it still struggles to keep the whole story straight. The paper found that the hardest rule to follow was C6: matching the exact number of items in the menu. If the menu has two Spanish restaurants, the AI often guesses three or says "many," failing the math check.

What the Paper Rules Out

The authors are careful to say what this method is not.

It is not a magic wand that automatically rewrites the AI's bad sentences into perfect, fluent English. The system can point out the error and suggest a fix (like "change 'three' to 'two'"), but it doesn't automatically generate the new, smooth sentence for you. That's a job for future work.
It is not a method that works perfectly on any conversation without setup. The system needs to know the "slots" (like Food, Price, Area) and the specific menu for that domain. It's not a general "common sense" checker; it's a specific fact-checker for specific topics.

The Bottom Line

The paper concludes that treating dialogue consistency as a logic puzzle is a powerful way to catch AI hallucinations. It suggests that while Large Language Models are great at sounding human, they are still shaky at sticking to the facts. By using a CSP solver, we can catch about 75.9% of these inconsistencies automatically.

The authors admit their results are based on controlled experiments and specific datasets, so we don't know yet how this holds up in the messy, unpredictable real world. But for now, it's a solid step toward making sure our AI chefs don't serve us imaginary meals.

Technical Summary: Towards Detecting Inconsistencies in End-to-end Generated TODs

Problem Statement

Generative AI, particularly Large Language Models (LLMs), is shifting Task-Oriented Dialogue (TOD) systems from component-based pipelines to end-to-end approaches. While this simplifies development, LLMs frequently generate inconsistencies when adhering to specific domain knowledge bases (KBs). These inconsistencies manifest as domain hallucinations (e.g., citing non-existent entities or incorrect counts) and dialogic incoherence (e.g., contradicting previous turns). Such errors undermine the reliability of TODs in real-world applications. The paper posits that detecting these inconsistencies requires analyzing the entire dialogue context, as local coherence does not guarantee global consistency with the KB.

Methodology

The authors propose modeling dialogue consistency as a Constraint Satisfaction Problem (CSP). The core intuition is that a consistent dialogue corresponds to a valid solution within a CSP defined by the dialogue's variables and the domain's constraints.

1. Conceptual Framework

The approach categorizes consistency constraints into three types:

Linguistic Constraints: Ensure morpho-syntactic correctness (e.g., preposition usage matching slot types like Area vs. Price).
Dialogic Constraints: Maintain semantic coherence across turns (e.g., ensuring a system response aligns with the user's prior request) and prevent redundancy within a single utterance.
Domain Constraints: Enforce alignment with the Knowledge Base. This includes verifying the existence of entities, the correctness of slot values, and the accuracy of instance counts (e.g., "There are three restaurants" must match the actual count in the KB).

2. The CSP Pipeline

The methodology operates in four distinct steps (illustrated in Figure 2 of the paper):

Variable Identification: The system identifies "variables" within the dialogue. These are text segments referencing slot values (e.g., "Spanish") or instance counts (e.g., "three"). Variables can be identified via existing annotations (e.g., MultiWOZ) or automatically using GPT-4o.
Constraint Definition: The system instantiates six domain-independent constraint patterns ( $C_1$ through $C_6$ ) based on the identified variables and the provided KB. These patterns cover semantic validity, turn-to-turn coherence, and KB alignment (existence, non-existence, and exact counts).
CSP Solving: A CSP solver (specifically Chuffed via the MiniZinc language) computes all valid assignments for the variables that satisfy the defined constraints.
Consistency Assessment: The original dialogue's variable assignments are compared against the set of valid solutions found by the solver.
- If the dialogue matches a valid solution, it is consistent.
- If no solution exists or the dialogue does not match any solution, the system identifies the inconsistency.
- The solver can suggest minimal changes (e.g., changing "three" to "two") to transform the dialogue into a valid solution.

Key Contributions

CSP Formulation: The paper introduces a novel approach to modeling TOD consistency as a Constraint Satisfaction Problem, a fully original contribution to the field.
Reusable Experimental Setting: The authors establish a pipeline for automatically evaluating TOD consistency against a CSP solver, capable of handling both annotated and LLM-extracted variables.
Empirical Validation: The study demonstrates that CSP-based detection is effective, achieving high accuracy in identifying inconsistencies.

Experimental Results

The authors conducted experiments using the MultiWOZ 2.3 dataset and various LLMs.

Inconsistency Detection Performance

The authors evaluated the ability of the CSP pipeline to detect inconsistencies in dialogue-KB pairs:

Upper Bound: Using manual MultiWOZ global annotations with the CSP solver achieved 91.6% accuracy.
Local Evaluation: Applying constraints locally (per turn) dropped accuracy to 79.0%, highlighting the necessity of global context.
End-to-End Pipeline: Using GPT-4o for automatic variable extraction and the CSP solver for evaluation achieved 75.9% accuracy. The authors note that errors in this setting were primarily due to imperfect variable extraction by the LLM rather than the CSP logic itself.

LLM Re-lexicalization Analysis

In a separate experiment, the authors tasked four models (LLaMA-3.1 8B, GPT-3.5-Turbo, GPT-4o, and GPT-o1) with re-lexicalizing de-lexicalized dialogues using a provided KB.

Performance: Even the best models (GPT-4o and GPT-o1) achieved low Global Consistency Accuracy (GCA) of 0.14 and Variable Consistency Accuracy (VCA) of roughly 0.42.
Findings: This indicates that despite their generative capabilities, LLMs struggle significantly with strict adherence to domain constraints. The analysis revealed that $C_6$ (exact instance count matching the KB) was the most critical constraint for consistency, followed by $C_1$ (slot value validity).
Baselines: Simple baselines (random assignment or frequency-based) performed poorly, confirming that the task requires genuine reasoning rather than statistical guessing.

Significance and Claims

The paper claims that while LLMs are powerful, they are not inherently reliable for task-oriented domains requiring strict factual adherence. The significance of this work lies in:

Detection Capability: Providing a robust, automated method to detect inconsistencies that traditional metrics (like BLEU or ROUGE) miss.
Diagnostic Utility: The CSP approach does not just flag errors but identifies the specific variable assignments causing the failure and suggests minimal corrections.
Limitations of LLMs: The study empirically demonstrates that even advanced models (GPT-4o, GPT-o1) fail to maintain consistency in roughly 86% of cases at the dialogue level when re-lexicalizing, underscoring the need for external verification mechanisms like CSP.

The authors conclude that CSP-based methodologies offer a promising avenue for evaluating and improving automated dialogue generation, particularly in applications with strong domain knowledge requirements. They acknowledge limitations, noting that the system currently detects and suggests corrections but does not automatically regenerate fluent natural language responses, and that future work is needed to test robustness on organically generated dialogues.