Technical Summary: IDP-Bench

Problem Statement

As Large Language Models (LLMs) are increasingly deployed as personal AI assistants with access to sensitive user data, ensuring privacy has become a critical safety challenge. While prior research has extensively benchmarked LLMs on individual-level privacy risks, these evaluations often overlook Interdependent Privacy (IDP). IDP describes scenarios where one person's privacy is contingent on the data-sharing decisions of others, meaning personal data is often co-owned rather than belonging to a single individual.

In real-world sociotechnical systems, LLM agents frequently process data involving multiple subjects (e.g., meeting notes, group chats, contact lists, shared photos). Current benchmarks, such as CI-Bench and PrivacyLens, primarily focus on single-subject scenarios or do not formally distinguish between multiple individuals involved in the same data flow. Consequently, there is a gap in understanding whether LLMs can reason about the complex privacy norms required when data involves co-implicated parties who may not have consented to or even be aware of the data flow.

Methodology

The authors introduce IDP-Bench, the first benchmark specifically designed to evaluate LLMs' ability to reason about interdependent privacy scenarios. The methodology is grounded in the Contextual Integrity (CI) framework and IDP theory, structured into three main components: a risk model, a data generation pipeline, and a multi-level evaluation protocol.

1. Risk Model

The benchmark defines a privacy risk scenario involving four components and one information object:

• Data Sender ($u$): The individual initiating the sharing.
• Data Recipient ($r$): The intended target of the sharing.
• Primary Data Subject ($d_p$): The main individual the data concerns.
• Secondary Data Subjects ($\{d_{s1}, \dots, d_{sk}\}$): A non-empty set of additional individuals whose information is simultaneously exposed in the data but who may not have consented.
• Transmission Principle ($tp$): The method of transmission.

The core challenge is determining if an LLM can correctly identify that data $D$ is multi-subject, recognize all co-implicated individuals, and judge the appropriateness of the information flow given the absence of consent from secondary subjects.

2. Data Generation Pipeline

The benchmark utilizes a synthetic pipeline to generate 100 unique scenarios (97 used after filtering), consisting of:

• Seed Extension: Starting from 493 PrivacyLens seeds (5-tuples), the authors used Qwen2.5-72B-Instruct to extend them into 9-field IDP seeds. This involved adding fields for secondary subjects, interdependence type (co-owned, correlated, familial, co-location), and interdependence description.
• Verification: A unit-test verifier checks structural and semantic validity, ensuring secondary subjects are distinct from the primary subject and that the scenario realistically reflects everyday IDP contexts (e.g., social media, group chats) rather than specialized professional settings.
• Vignette Generation: Verified seeds are expanded into 7-sentence narratives. The first six sentences establish actors and joint data without explicit labels, while the seventh sentence (withheld during evaluation) explicitly states the privacy leakage to secondary subjects.
• Underspecified Instructions: Each vignette is paired with a high-level sharing instruction that omits specific details about what to share or who is involved, forcing the model to infer privacy constraints.

3. Evaluation Protocol

The evaluation probes eight open-source LLMs (Qwen2.5 family and Llama-3.1 family) across three progressive reasoning levels using two independent LLM judges (Llama-3.1-70B and Qwen2.5-72B):

• Level 1: Context Understanding (Q1–Q7): Extracting CI parameters (information attribute, primary/secondary subjects, sender, recipient, transmission principle) and identifying what information about secondary subjects would be revealed.
• Level 2: Co-ownership Recognition (Q8): Determining if the data is co-owned by multiple individuals.
• Level 3: Appropriateness Judgment (Q9): Deciding if the sharing is appropriate and providing a justification.

The evaluation uses five prompt variants per question to measure sensitivity to phrasing and employs an LLM-as-a-judge approach with specific prompts for semantic matching, partial credit, and strict exact matching.

Key Contributions

1. Extension of CI Seeds: The authors formalized the transition from single-subject to multi-subject seeds by extending the PrivacyLens 5-tuple format to include secondary subjects and interdependence descriptors.
2. IDP-Bench Construction: A synthetic benchmark of 100 IDP scenarios covering diverse multi-subject data types (meeting notes, group photos, contact lists, etc.) anchored in realistic digital platforms.
3. Comprehensive Evaluation: A multi-level assessment of open-source LLMs across context understanding, co-ownership recognition, and appropriateness judgment, comparing performance across model scales and families.

Results

The evaluation of eight models (ranging from 1.5B to 72B parameters) yielded the following findings:

• Co-ownership Recognition (Level 2): Models perform strongly here. Six out of eight models scored above 90% on Q8, with larger models (e.g., Llama-3.1-70B, Qwen2.5-72B) reaching near-ceiling performance. This suggests models can reliably identify the presence of co-ownership when explicitly prompted.
• Context Understanding (Level 1): Performance is mixed and generally lower.
  • Sender/Recipient Identification: Generally high for larger models (70–90%), but smaller models (e.g., Qwen2.5-1.5B) struggle significantly (38–40%).
  • Primary/Secondary Subject Identification: This remains a significant challenge. While primary subject identification (Q2) improves with scale, secondary subject identification (Q3) shows inconsistent scaling; for instance, Qwen2.5-14B outperformed Qwen2.5-72B in some judge configurations.
  • Information Attribute (Q1): All models scored below 76%, indicating difficulty in classifying the core data type.
• Appropriateness Judgment (Level 3): Performance improves with model scale. The two largest Qwen models (32B, 72B) achieved 91–96% accuracy, while smaller models (1.5B, 3B, 7B) remained in the 60–71% range. Notably, large models often reach the correct normative conclusion (sharing is inappropriate) even when they fail to correctly identify the specific secondary subjects or information attributes underlying that conclusion.
• Prompt Sensitivity: Performance varies significantly across prompt variants, particularly for IDP-specific questions, highlighting a lack of robustness in reasoning about these scenarios.

Significance and Claims

The paper claims that IDP-Bench reveals a critical gap in current LLM privacy capabilities: while models can recognize that data is co-owned, they struggle with the finer-grained reasoning required to identify who is implicated and what specific information is at risk.

The authors emphasize that:

• Scale is not a panacea: While appropriateness judgment improves with model size, smaller models and even some larger ones fail to consistently resolve the detailed IDP parameters (secondary subjects) necessary for robust privacy protection.
• Current benchmarks are insufficient: Existing privacy benchmarks that focus on single-subject scenarios fail to capture the relational nature of real-world data, potentially leading to a false sense of security regarding LLM privacy.
• Future Directions: The work underscores the need for more targeted study of IDP in LLM privacy research, including extending evaluations to agentic settings where models perform actions rather than just answering questions, and developing training objectives that account for the multi-subject nature of data.

The authors remain modest, noting that their current evaluation is reasoning-based and does not yet include action-based settings where models use tools to execute privacy-violating actions, which they identify as a necessary future extension.