💬 NLP

Distilling Bayesian Belief States into Language Models for Auditable Negotiation

The paper introduces BOND, a framework that distills an LLM-based Bayesian teacher's explicit opponent-belief inference into a smaller 8B student model, achieving superior negotiation performance and auditable belief tracking compared to state-of-the-art baselines.

Original authors: Zongqi Cui, Baihan Lin

Published 2026-05-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zongqi Cui, Baihan Lin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a high-stakes poker game. You see a player make a move, but you have no idea why they made it. Did they bluff because they thought you were weak? Did they fold because they knew they had a losing hand? Or were they just guessing?

In the world of AI negotiation, Large Language Models (LLMs) are like those poker players. They can talk and make deals very fluently, but their "thought process" is hidden inside a black box. We can't see what they believe about the other person's priorities, making it hard to trust or debug their decisions.

This paper introduces a new system called BOND (Bayesian Opponent-belief Negotiation Distillation) to solve this problem. Here is how it works, using simple analogies:

1. The Problem: The "Black Box" Negotiator

Current AI negotiators are like magicians. They pull a rabbit out of a hat (make a deal), but you can't see the rabbit or the hat. If the AI says, "I'll give you the firewood," you don't know if it thinks you love firewood, or if it's just repeating what it heard before. This makes the AI "unauditable"—you can't check its math.

2. The Solution: The "Teacher-Student" Team

The authors created a two-part system to make the AI's thinking visible.

The Teacher (The Mathematician): This is a smart AI that acts like a detective. It listens to the conversation and asks, "Based on what the other person said, what is the probability they care most about Food, Water, or Firewood?"
- Instead of just guessing, the Teacher uses a mathematical method (Bayesian inference) to update its beliefs after every single sentence. It keeps a running scorecard of the six possible ways the opponent might prioritize items.
- The Result: The Teacher knows exactly what it thinks the opponent wants, and it can show you that scorecard.
The Student (The Actor): The Teacher is great at math but slow and expensive to run. So, the authors "distilled" (compressed) the Teacher's brain into a smaller, faster AI called the Student.
- The Student learns to act like the Teacher. But here is the magic trick: When the Student speaks, it doesn't just say, "I accept your offer." It also whispers its internal scorecard to the world.
- It outputs a tag like: <posterior> I am 80% sure you want Water, 15% Firewood, 5% Food </posterior>.
- This makes the AI's belief state auditable. You can see exactly what it thought before it acted.

3. The Experiment: The Campsite Game

To test this, they used a dataset called CaSiNo, which simulates two people negotiating over a campsite. They have to split three items: Food, Water, and Firewood.

Each person secretly has a priority list (e.g., "I need Water most, then Firewood, then Food").
The AI's job is to figure out the other person's secret list and make a fair deal.

4. The Results: Smaller is Better for Transparency

The paper found some surprising things:

The Teacher was very good at guessing the opponent's priorities (a "Brier score" of 0.085, which is very low and good).
The Student (the smaller, faster model) learned to copy the Teacher's thinking very well. It achieved a score of 0.114.
The Big Competitor: They compared this to a massive 70-billion-parameter AI (the "70B baseline"). While the big AI was slightly better at making the right final deal, it was terrible at explaining why. Its internal guesses were confused and uncalibrated (score of 0.194).
The Takeaway: The smaller 8-billion-parameter Student is the winner for transparency. It is small enough to run easily but smart enough to show you its "workings" clearly.

5. The Catch: The "Whisper" vs. The "Action"

The paper also found a small flaw. While the Student is great at reporting what it believes, it doesn't always act based on those beliefs.

Imagine a student who raises their hand to answer a question correctly (the belief) but then writes the wrong answer on the test (the action).
The authors found that the AI learned to "whisper" its beliefs accurately, but the connection between that whisper and its final decision was sometimes weak. It's like the AI learned to fill out a report card perfectly, but sometimes forgot to use that report card to guide its next move.

Summary

BOND is a framework that forces an AI negotiator to show its homework. Instead of just giving you a deal, it tells you, "I think you want X, so I am offering Y."

Why it matters: It turns a mysterious "black box" AI into a transparent partner where you can see its beliefs, check if they are wrong, and understand why it made a mistake.
The Limit: The paper admits that while the AI is now transparent, it hasn't fully mastered the art of using those beliefs to make perfect decisions yet. It's a step toward "auditable" AI, where we can trust the system because we can see its logic, not just its results.

Technical Summary: Distilling Bayesian Belief States into Language Models for Auditable Negotiation

Problem Statement

Negotiation agents require the ability to infer an opponent's hidden values, update these beliefs over dialogue turns, and select actions under uncertainty. While end-to-end Large Language Models (LLMs) can generate fluent negotiation dialogue, their internal representations of opponent beliefs are typically implicit and opaque. This lack of transparency makes it difficult to distinguish whether an agent's action stems from a stable, updated model of the opponent or from shallow dialogue imitation. Conversely, traditional Bayesian cognitive models offer explicit priors, likelihoods, and posteriors but often rely on hand-designed likelihood functions that do not scale to free-form, strategic negotiation language. The core challenge is to combine the linguistic flexibility of LLMs with the interpretability of Bayesian inference to create a negotiation agent that is both effective and auditable.

Methodology: The BOND Framework

The authors propose BOND (Bayesian Opponent-belief Negotiation Distillation), a modular framework that separates language interpretation, belief revision, and action selection. The system operates in the CaSiNo negotiation domain, where two participants negotiate over three items (Food, Water, Firewood) with private priority orderings. The hypothesis space consists of six possible priority orderings ( $\Theta$ , where $|\Theta|=6$ ).

1. The Bayesian Teacher

The teacher agent functions as a "likelihood estimator" rather than a black-box policy. Its architecture involves four stages:

Language Scoring: An LLM (Llama-3.1-8B-Instruct with LoRA) scores the compatibility of the observed dialogue context against each of the six candidate opponent priority orderings. These scores serve as likelihood-like signals.
Bayesian Update: A Bayesian module converts these scores into a posterior distribution over the six orderings using the formula:
$p_t(\theta) = \frac{\tilde{L}_t(\theta)p_{t-1}(\theta)}{\sum_{\theta' \in \Theta} \tilde{L}_t(\theta')p_{t-1}(\theta')}$
where $\tilde{L}_t$ represents the temperature-scaled and clipped likelihood score. The prior is initially uniform.
Menu Scoring: The agent enumerates feasible allocations and scores them based on its own utility and the expected utility of the opponent, marginalized over the posterior distribution.
Decision: The agent selects an action (accept, reject, counter-offer, etc.) based on the menu scores.

2. Distillation to the Student

The authors distill the Bayesian teacher into a smaller 8B student model. Unlike standard end-to-end agents, the student is trained to emit a structured, tagged output that includes:

<posterior>: A normalized probability distribution over the six opponent priority orderings.
<selected_intent>: The chosen action (e.g., submit, accept, reject).
<selected_content>: The specific allocation if applicable.
<utterance>: The natural language response.

This design externalizes the belief state, allowing it to be scored and inspected at every turn.

3. Evaluation Protocol

The system is evaluated on a held-out set of 150 CaSiNo dialogues (1,054 turn-level predictions). Key metrics include:

Posterior Brier Score: Measures the calibration of the predicted probability distribution against the true opponent ordering.
Accept Prediction (F1): Accuracy of accept/reject decisions.
Bid Similarity: Cosine similarity between predicted and human allocations.
Auditability Diagnostics: Decomposition of errors into "belief errors" (wrong posterior) vs. "policy errors" (wrong action given the posterior) and interventions where the posterior tag is forcibly replaced.

Key Results

Posterior Calibration and Distillation

Teacher Performance: The Bayesian teacher achieves a mean Brier score of 0.085 over the six opponent-priority hypotheses, significantly outperforming the uniform reference of $5/36 \approx 0.139$ .
Student Performance: The distilled 8B student preserves much of this signal, achieving a Brier score of 0.114. This is also well below the uniform reference, indicating the student successfully learns to emit calibrated belief states.
Comparison with 70B Baseline: A 70B structured Chain-of-Thought (CoT) baseline achieves higher Accept-F1 (0.947 vs. 0.908) but fails to provide a calibrated posterior. When forced to emit a posterior via self-consistency, the 70B baseline achieves a Brier score of 0.194, which is worse than the uniform reference and significantly worse than the 8B student.

Auditability and Error Decomposition

Separation of Errors: Analysis of 181 decision turns reveals that belief errors and action errors are separable. For instance, 67 turns had the correct MAP (Maximum A Posteriori) belief but an inconsistent action, while 27 turns had a wrong belief but an action consistent with that wrong belief.
Intervention Limits: When the posterior tag was forcibly replaced with the ground truth (correct-prefix intervention) or an incorrect value (adversarial-prefix intervention), the student's action changed in only 1.5% to 1.8% of cases. This suggests that while the student learns to report beliefs accurately, the causal coupling between the reported posterior and the action selection (the "posterior $\to$ planner" dependency) is weaker in the distilled student than in the teacher.

Ablation Findings

Full-Context vs. Incremental: The teacher relies on scoring the full dialogue context. An incremental approach (scoring only the latest utterance) failed, producing posteriors worse than uniform, indicating that the LLM's attention mechanism is crucial for temporal evidence integration.
Role of Tags: Removing the posterior tag during training (Action-only student) resulted in slightly better decision metrics (Accept-F1 0.921), confirming that the posterior tag's primary value is inspectability, not decision performance.

Significance and Claims

The paper does not claim that the distilled student is the superior negotiation policy in terms of winning outcomes; in fact, the 70B baseline outperforms it on decision metrics. Instead, the significance of BOND lies in auditability:

Externalized Belief States: The framework successfully distills a valid, scoreable belief interface into a smaller model. The student exposes a calibrated posterior distribution at every turn, allowing researchers to distinguish between "wrong-belief" failures and "right-belief/wrong-action" failures.
Transparency over Black-Box Performance: The work demonstrates that a smaller model (8B) can expose a valid belief state that is better calibrated than a much larger model (70B) using structured prompting.
Limitations of Current Distillation: The authors modestly note that standard Supervised Fine-Tuning (SFT) distillation transfers the reporting of beliefs more effectively than the causal use of those beliefs for decision-making. The weak belief-action coupling in the student suggests that future auditable agents will require constrained decoding or auxiliary planning losses to ensure actions are strictly conditioned on the reported beliefs.

In summary, BOND provides a method to make the hidden opponent model of a negotiation agent explicit, distillable, and scoreable, offering a new diagnostic lens for understanding where and why negotiation agents fail.