💻 computer science

A Speaker-Aware Hybrid Framework for Faithful Dialogue Summarization via Parameter-Efficient Reinforcement Learning

This paper proposes a parameter-efficient, speaker-aware hybrid framework that combines hierarchical extractive condensation, LoRA-based adaptation, and reinforcement learning with a novel speaker-attribution reward to significantly improve faithfulness and reduce hallucinations in dialogue summarization while maintaining competitive lexical quality.

Original authors: Nidhi Passi, Nitin Arvind Shelke

Published 2026-08-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Nidhi Passi, Nitin Arvind Shelke

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are sitting in a crowded coffee shop, trying to write a summary of a conversation between three friends. It's chaotic: people talk over each other, use slang, and constantly switch topics. Now, imagine you have a super-smart robot assistant to help you write that summary. The problem is, this robot is a bit of a gossip. It's great at picking out the juicy details, but it keeps mixing up who said what. It might write, "Mark said he was buying the cake," when actually, "Hannah" was the one who offered. In the world of artificial intelligence, this is called a "hallucination," and it's a major headache for anyone trying to use AI to summarize meetings, customer service chats, or group texts. If the summary gets the facts wrong, it's not just unhelpful; it's misleading.

To fix this, scientists are teaching AI models to be better listeners. They use techniques like "reinforcement learning," which is like training a dog with treats: the AI gets a "reward" when it does something right (like keeping the facts straight) and a "time-out" when it messes up. They also use "parameter-efficient fine-tuning," a fancy way of saying they tweak only a tiny, specific part of the robot's brain instead of rebuilding the whole thing, saving massive amounts of energy and time. The big question is: can we make these robots not just summarize well, but summarize faithfully, ensuring that every action and statement is correctly attributed to the right person, without needing a supercomputer to do it?

This paper introduces a clever new framework designed to solve exactly that problem. The researchers, Nidhi Passi and Nitin Arvind Shelke, built a three-step system that acts like a highly organized editorial team for dialogue summaries. First, they use a "hierarchical attention" mechanism that acts like a spotlight. Instead of reading every single word of a long, messy chat, this spotlight scans the conversation to find the most important sentences, but with a twist: it pays extra attention to who is speaking. It filters out the "Hey, how are you?" small talk and keeps the core facts, tagging them with the correct speaker's name.

Next, this filtered, speaker-tagged information is fed into a language model called FLAN-T5. However, instead of retraining the entire massive model (which would be like rewriting the whole dictionary just to learn a new word), they use a technique called LoRA. Think of LoRA as adding a small, specialized set of sticky notes to the model's existing knowledge. These notes teach the model how to handle dialogue specifically, updating only about 1.8 million parameters out of the model's 250 million. This makes the process incredibly fast and efficient.

Finally, and perhaps most importantly, they use a reinforcement learning method called Proximal Policy Optimization (PPO) to refine the summary. Imagine a strict editor who reads the draft and checks every single fact against the original chat. This editor doesn't just check if the summary sounds good; it breaks the summary down into tiny claims (like "Mark bought the cake") and verifies them against the specific lines spoken by Mark. If the summary says "Hannah bought the cake," the editor gives a big penalty. This "speaker-attributed faithfulness" reward forces the AI to learn that getting the speaker right is just as important as getting the words right.

The results are impressive. When tested on standard dialogue datasets like SAMSum and DialogSum, this new framework managed to produce summaries that were just as fluent and readable as those from much larger, fully trained models. But the real magic happened in the facts. The system improved its "faithfulness" score by a significant margin—specifically, it boosted a metric called HHEM-2.1 by 0.14 and a speaker-attribution score by 0.16 compared to the strongest baseline (BART-large). Even more striking, it achieved all of this while updating only 1.8 million parameters, a tiny fraction of what other methods require.

The paper explicitly argues against the idea that simply making models bigger or using standard training methods is enough to fix these errors. The authors show that without explicitly optimizing for speaker attribution, even the smartest models will continue to mix up who said what. They also demonstrate that while instruction-tuned large language models are powerful, they still struggle with these specific attribution errors in zero-shot settings. By combining a smart extraction step, a lightweight adaptation method, and a strict fact-checking reward system, this framework offers a more efficient and accurate way to summarize conversations. It suggests that the key to faithful AI isn't just raw power, but a structured approach that respects the identity of every speaker in the room.

Technical Summary: A Speaker-Aware Hybrid Framework for Faithful Dialogue Summarization via Parameter-Efficient Reinforcement Learning

1. Problem Statement

Dialogue summarization is critical for applications such as meeting analysis, customer support, and conversational intelligence. However, existing abstractive summarizers (e.g., BART, PEGASUS, T5) frequently generate factually inconsistent summaries, particularly regarding speaker attribution. In multi-speaker dialogues, models often misattribute actions or statements to the wrong speaker. These errors constitute a dominant source of hallucination, affecting over 35% of test instances in benchmarks like SAMSum.

Current limitations include:

Speaker Agnosticism: Hybrid extract-then-rewrite methods often ignore speaker identity during content selection, treating speakers as ordinary tokens rather than distinct conversational roles.
Optimization Mismatch: Standard supervised fine-tuning (SFT) optimizes token-level cross-entropy, which correlates poorly with sequence-level factual consistency and fails to explicitly penalize speaker misattribution.
Computational Cost: Fully fine-tuning large models is computationally expensive and prone to catastrophic forgetting, while instruction-tuned Large Language Models (LLMs) often fail to outperform fine-tuned task-specific models in zero-shot settings and face privacy/computational constraints.

2. Methodology

The authors propose a three-stage hybrid framework that integrates hierarchical extractive condensation, parameter-efficient adaptation, and reinforcement learning (RL) to optimize speaker-attributed faithfulness.

Stage 1: Training-Free Hierarchical Semantic Condensation

To address the dispersion of salient information across short, multi-turn dialogues, the framework employs a training-free hierarchical attention mechanism:

Token-Level Attention: A frozen pretrained sentence encoder generates contextual token embeddings. Tokens are aggregated into utterance vectors using attention weights derived from the global dialogue context.
Utterance-Level Attention: A salience score is computed for each utterance by combining its similarity to the global dialogue context and its similarity to the specific speaker's context.
Selection: The top- $K$ utterances are retained in their original temporal order to form an extractive condensate ( $d^{ext}$ ), limited to 60% of the original dialogue length. This stage introduces no trainable parameters.

Stage 2: Parameter-Efficient Abstractive Rewriting (LoRA)

The condensed dialogue is rewritten using a FLAN-T5-base encoder-decoder model adapted via Low-Rank Adaptation (LoRA):

Efficiency: Only the LoRA adapter matrices ( $A$ and $B$ ) are trained, updating approximately 1.8M parameters (less than 0.8% of the 250M backbone), while the pretrained weights remain frozen.
Supervised Fine-Tuning (SFT): The model is initially trained using teacher-forced cross-entropy loss to learn fluent summaries from the condensates. This model serves as the reference policy ( $\pi_{ref}$ ) for the subsequent RL stage.

Stage 3: Speaker-Attributed Faithfulness Optimization via PPO

To bridge the faithfulness gap, the framework employs Proximal Policy Optimization (PPO) with a novel composite reward function:

Reward Components: The reward $R$ $R$ combines four terms:
1. Lexical Quality ( $R_L$ ): ROUGE-L F1 score.
2. Semantic Similarity ($BS$): BERTScore F1.
3. Speaker-Attributed Faithfulness ( $Fact_{spk}$ ): A novel objective where the generated summary is decomposed into atomic claims $\langle speaker, predicate, object \rangle$ . Each claim is verified against the corresponding speaker's turns in the source dialogue using the HHEM-2.1 entailment model.
4. Brevity Penalty ( $\phi$ ): A soft length penalty to prevent overly long or short outputs.
Optimization: The PPO objective maximizes this composite reward while constraining the updated policy to remain close to the SFT reference policy via a Kullback–Leibler (KL) regularization term. This prevents "reward hacking" and preserves fluency.

3. Key Contributions

The paper outlines four primary contributions:

Unified Framework: A speaker-aware hybrid pipeline integrating hierarchical extractive condensation, LoRA-based parameter-efficient adaptation, and KL-regularized PPO.
Novel Reward Objective: A speaker-attributed faithfulness reward that verifies factual tuples against individual speaker turns rather than the dialogue as a whole, enabling explicit optimization of speaker-level consistency.
Parameter-Efficient Strategy: A reinforcement learning strategy that achieves sequence-level optimization while updating fewer than 0.8% of the backbone parameters.
Empirical Validation: Extensive experiments demonstrating that the framework improves faithfulness metrics significantly while maintaining lexical quality comparable to fully fine-tuned baselines, using only 1.8M trainable parameters.

4. Experimental Results

The framework was evaluated on the SAMSum (informal chat) and DialogSum (task-oriented) benchmarks.

Performance on SAMSum

The proposed method achieved the following scores:

ROUGE-1/2/L: 53.0 / 28.2 / 49.0 (comparable to BART-large).
BERTScore: 0.923.
HHEM-2.1 (Faithfulness): 0.86 (an improvement of 0.14 over BART-large).
Speaker-Attributed Faithfulness ( $Fact_{spk}$ ): 0.84 (an improvement of 0.16 over BART-large).
Parameters: Only 1.8M trainable parameters were updated.

Cross-Dataset Generalization

Evaluated on DialogSum without further fine-tuning, the model achieved a ROUGE-L of 42.7 and improved HHEM-2.1 from 0.69 (BART-large) to 0.83, demonstrating effective transferability.

Ablation and Error Analysis

Ablation: Removing the speaker-attributed faithfulness reward ( $Fact_{spk}$ ) caused HHEM-2.1 to drop from 0.86 to 0.79, confirming its critical role. Removing PPO refinement also degraded faithfulness significantly.
Error Reduction: The framework reduced wrong-reference errors (incorrect speaker attribution) from 31% in BART-large outputs to 9%.
Efficiency: Inference latency was 0.31s on an NVIDIA A100, faster than full fine-tuning (0.42s) and BART-large (0.55s), due to input condensation and merged-weight inference.

5. Significance and Claims

The authors claim that explicitly optimizing speaker-level factual consistency through reinforcement learning offers an effective and parameter-efficient approach to improving dialogue faithfulness.

Beyond Lexical Overlap: The results demonstrate that high ROUGE scores do not guarantee factual consistency; explicit optimization of speaker attribution is necessary to reduce hallucinations.
Efficiency vs. Performance: The study challenges the notion that larger models or full fine-tuning are required for high-quality summarization, showing that a principled integration of extraction, LoRA, and speaker-aware RL can outperform fully fine-tuned baselines in faithfulness while using a fraction of the parameters.
Limitations: The authors modestly note that the extractive condensation stage may occasionally omit secondary discussion points in long, multi-topic conversations. Future work is suggested to address adaptive utterance selection and multilingual expansion.

In conclusion, the paper posits that the proposed framework provides a robust, computationally efficient solution for faithful multi-speaker dialogue summarization by directly addressing the root cause of attribution errors through a novel reinforcement learning objective.