Technical Summary: MuPPET

Problem Statement

Large Language Model (LLM) agents are increasingly deployed in multi-party environments, such as group chats in team collaboration tools. In these settings, an agent acting on behalf of a user must navigate complex social contexts where a single disclosure of private information reaches all group members simultaneously.

Existing contextual privacy benchmarks, grounded in the theory of Contextual Integrity (Nissenbaum, 2004), evaluate whether models respect social norms regarding information flow. However, these benchmarks (e.g., ConfAIDE, PrivacyLens, CI-Bench) are limited to one-to-one interactions (single user querying a model or a single organization receiving output). They fail to account for the structural complexity of multi-party settings, where privacy requires that every piece of information be appropriate for every recipient in the group. Consequently, the privacy risks of LLM agents in multi-party scenarios remain unmeasured, and prior evaluations may systematically underestimate leakage.

Methodology: MuPPET Benchmark

The authors introduce MuPPET (Multi-Party Privacy Exposure Testing), a benchmark designed to evaluate contextual privacy in multi-party conversations.

Dataset Construction

• Scale: 562 multi-party conversations in English.
• Scenario: Workplace environments involving 11 types of work departments (e.g., Engineering, Healthcare).
• Structure:
  • Seeds: Manually curated scenarios containing a work topic, a sensitive attribute (e.g., health, family status), and a legitimate managerial question that could inadvertently elicit sensitive info.
  • User Groups: Synthetic workers generated from the PANORAMA dataset, organized into teams of 20. One user is designated as the "target" for whom the LLM acts.
  • Memories: The LLM is provided with background information (memories) about the target user, including sensitive details revealed in prior one-on-one interactions with colleagues.
  • Conversation: Multi-turn group chats generated by Gemini 2.5 Flash, ending with a question directed at the assistant.
• Evaluation Logic: The model must answer the group question using its memories. A privacy leak occurs if the response reveals private information to recipients for whom disclosure is inappropriate, even if no explicit sharing preference was given.

Metrics

1. Privacy Leakage: Measured using an LLM-as-a-judge. The judge checks if the model's response reveals specific private details from the source memories, either directly or through strong semantic implication.
2. Utility: Measured by whether the model successfully extracts and applies the user's preferences and constraints (e.g., avoiding heavy workload) from the memories without necessarily disclosing the underlying sensitive cause.

Experimental Setup

The benchmark evaluated:

• Open-weight models: Meta-Llama 3/3.1 (8B), Qwen 3 (4B, 8B, 14B).
• Frontier closed models: GPT-5.5, Gemini 2.5 Pro.
• Defence Strategies:
  • CI-Mem: Augmenting system prompts with privacy instructions (Low/Medium/High enforcement).
  • PrivacyChecker: Decomposing privacy reasoning into explicit steps within the prompt.

Key Results

1. Multi-Party vs. One-to-One Leakage

MuPPET demonstrates that multi-party settings represent a significantly higher risk than one-to-one evaluations.

• Leakage Increase: All tested models leaked more in multi-party settings than in equivalent one-on-one scenarios. For example, Llama 3 8B Inst. leaked 45.68% in 1-on-1 but 57.14% in multi-party; Qwen3 14B jumped from 40.99% to 63.35%.
• Implication: Current benchmarks systematically underestimate privacy risks because they do not test the "party-tracking" burden required in group settings.

2. Model Performance and Vulnerability

• Open-Weight Models: Smaller models are the most vulnerable. Qwen3 8B showed the highest leakage (70.37%), and even the best open-weight models (Qwen3 4B, Llama 3 8B) leaked in over 50% of cases.
• Frontier Models: Proprietary models (Gemini 2.5 Pro, GPT-5.5) performed substantially better but still exhibited non-negligible leakage (e.g., Gemini 2.5 Pro at 23.46% in multi-party vs. 9.88% in 1-on-1).
• Utility-Privacy Trade-off: Models with higher utility scores (better at using memories to answer) tended to have higher leakage rates. Frontier models achieved the best balance, maintaining high utility with lower leakage than open-weight counterparts.

3. Effectiveness of Defences

• Partial Mitigation: Existing defences (CI-Mem and PrivacyChecker) reduced leakage but at a cost.
• Utility Degradation: Stronger privacy constraints (e.g., High enforcement) significantly reduced model utility.
• Scalability Issues: Smaller models struggled to follow complex privacy instructions (like PrivacyChecker), resulting in less effective protection compared to frontier models.
• Conclusion: Defences do not resolve the underlying party-tracking problem; they merely trade utility for partial protection.

4. Error Analysis

An analysis of leak causes revealed distinct failure modes based on model capability:

• Smaller Models: Leaks were primarily attributed to user tracking failures (inability to identify who is in the conversation).
• Reasoning Models: Leaks shifted to user-level contextual privacy failures (knowing who is in the chat but failing to decide if sharing with a specific person is appropriate).
• Frontier Models: Leaks were increasingly attributed to group-level privacy constraints or unexplained "Benchmark only" failures. This suggests a gap between knowledge and action: these models understand the roles and norms but still generate privacy-violating outputs in complex contexts.

Significance and Contributions

The paper claims the following contributions:

1. Introduction of MuPPET: A novel benchmark specifically for contextual privacy in multi-party conversations, addressing a gap left by all prior one-to-one benchmarks.
2. Identification of a Distinct Failure Mode: Evidence that multi-party leakage is a substantial, previously unmeasured risk where models fail to track multiple recipients and their respective privacy boundaries.
3. Vulnerability of Local Models: A finding that smaller open-weight models, often preferred for local deployment with sensitive data, are the most vulnerable to leakage, challenging the assumption that local deployment inherently ensures privacy.
4. Limitations of Current Defences: Demonstration that existing privacy defences offer only partial protection, degrade utility, and fail to solve the core party-tracking challenge.

The authors conclude that addressing multi-party leakage will likely require new modeling paradigms capable of handling complex information flows within shared contexts, rather than relying solely on prompt engineering or existing privacy filters.

Limitations

The authors acknowledge that MuPPET focuses on professional team settings, which may not capture the implicit norms of family or friend networks. The dataset is synthetically generated, which may lack the noise and ambiguity of real-world chats. Furthermore, the evaluation is English-centric, and privacy norms vary significantly across cultures and languages.