Learning Probabilistic Responsibility Allocations for Multi-Agent Interactions

1. Problem Definition

The paper addresses the challenge of modeling human behavior in multi-agent interactive settings (e.g., autonomous driving). While existing methods often focus on predicting what agents will do (trajectory forecasting), this work focuses on understanding why they deviate from their desired paths to maintain safety.

• Core Concept: Responsibility is defined as the degree to which an agent is willing to deviate from its desired control policy to satisfy shared safety constraints (e.g., avoiding collisions).
• The Gap: Previous work modeled responsibility as a deterministic mapping from scene context to a single responsibility value. However, human interactions are inherently multimodal (e.g., in a near-miss scenario, either agent could yield, or both could yield slightly). A deterministic model fails to capture this uncertainty and the variety of valid social interactions.
• The Challenge: There are no ground-truth labels for "responsibility" in real-world datasets. The model must learn the distribution of responsibility allocations indirectly, using only observed agent states and controls.

2. Methodology

The authors propose a Probabilistic Responsibility Model that learns a distribution over responsibility allocations conditioned on scene context. The framework integrates Control Barrier Functions (CBFs), Conditional Variational Autoencoders (CVAEs), and Transformer architectures.

A. Theoretical Foundation: Responsibility Safety Filter

The method builds upon Control Barrier Functions (CBFs) to ensure safety.

• Standard CBF: Ensures a single agent stays within a safe set by minimizing deviation from a desired control.
• Multi-Agent Extension: The authors extend this to $N$ agents by formulating a joint optimization problem (Problem 2).
  • Objective: Minimize the weighted sum of deviations from desired controls for all agents.
  • Responsibility Vector ($\gamma$): A vector where each element $\gamma_i$ represents the "cost" of deviation for agent $i$. A high $\gamma_i$ implies the agent is less willing to deviate (less responsible), while a low $\gamma_i$ implies high willingness to yield (more responsible).
  • Constraint: The optimization is subject to a set of pairwise CBF constraints ensuring collision avoidance between all agent pairs.

B. Learning Architecture: Responsibility CVAE

Since ground-truth $\gamma$ is unobserved, the authors use a Conditional Variational Autoencoder (CVAE) to learn the distribution $p(\gamma | s)$, where $s$ is the scene context.

• Latent Variable ($z$): A Gaussian latent variable captures the multimodal uncertainty of human interaction.
• Decoder Structure:
  1. Latent to Responsibility: The latent $z$ is passed through neural network layers to generate a responsibility vector $\gamma = g_\phi(z)$.
  2. Differentiable Optimization Layer: The generated $\gamma$ is fed into the Responsible Multi-Agent CBF Safety Filter (the optimization problem defined above). This layer is differentiable, allowing gradients to flow back to the neural network.
  3. Output: The filter produces the predicted control inputs $\hat{u}$.
• Training Objective: The model minimizes the Evidence Lower Bound (ELBO), effectively learning to generate $\gamma$ distributions such that the resulting safety-filtered controls $\hat{u}$ match the observed controls $u$ in the dataset.

C. Sequence Modeling: Transformer-Based CVAE

To handle complex, dynamic scenes with varying numbers of agents, the authors adapt the AgentFormer architecture:

• Input: A sequence of past agent states (position, velocity, type) and the current control.
• Mechanism: Uses self-attention and masking to handle agents entering/leaving the scene and to learn relationships between agents regardless of ordering (permutation invariance).
• Output: A distribution over responsibility allocations for the current time step, conditioned on the interaction history.

3. Key Contributions

1. Probabilistic Formulation: The first formulation of responsibility as a probability distribution rather than a point estimate, capturing the multimodal nature of human interactions.
2. Indirect Learning via Optimization: A novel architecture that learns unobserved responsibility variables by backpropagating through a differentiable optimization layer (the CBF safety filter), bypassing the need for labeled responsibility data.
3. Scalable Transformer Model: A sequence-to-sequence responsibility model capable of handling variable numbers of heterogeneous agents and complex interaction histories.
4. Interpretability: The model provides a mathematically grounded, quantitative measure of "who is responsible" in a collision scenario, offering insights beyond simple trajectory prediction.

4. Experimental Results

The method was evaluated on both synthetic data and the real-world INTERACTION driving dataset.

• Synthetic Data:

  • The model successfully reconstructed a bimodal ground-truth distribution of responsibility allocations.
  • It demonstrated that a Gaussian latent space could effectively capture the two distinct modes of interaction (e.g., Agent A yields vs. Agent B yields).

• INTERACTION Dataset (Real-World Driving):

  • Trajectory Prediction: The responsibility-induced trajectories achieved strong predictive performance (low Average Displacement Error and Miss Rate), significantly outperforming baselines that simply follow desired controls or predict controls directly without the safety filter.
  • Ablation Studies: The authors tested different activation functions for $\gamma$. They found that models allowing negative values (e.g., tanh or clip at -β) performed better than those constrained to $[0, 1]$, as real-world agents sometimes exhibit "negative responsibility" (aggressively pursuing their goal despite constraints).
  • Qualitative Insights:
    • The model correctly identified that an agent arriving later at an intersection (Green car) should have a lower responsibility value (higher willingness to yield) compared to an agent already in the intersection (Red car).
    • It captured bimodal behaviors: In some scenarios, the model predicted a distribution where the Green car could either speed up or slow down, reflecting the ambiguity of human decision-making.

5. Significance and Future Work

• Significance: This work shifts the paradigm from "predicting where agents go" to "understanding how agents negotiate safety." By quantifying responsibility, autonomous systems can become more socially compliant and trustworthy, as they can anticipate not just the trajectory, but the intent and willingness to yield of human drivers.
• Limitations: The real-world dataset was biased toward constant-velocity scenarios, limiting the diversity of learned responsibility outcomes. The current agent selection relies on distance heuristics rather than dynamic reachability.
• Future Directions:
  • Incorporating complex road geometry and visual context (signs, lights).
  • Using reachability-based methods to filter relevant agents more accurately.
  • Applying the model to counterfactual analysis (e.g., "What if the other agent had been more responsible?") to improve policy design for robots and autonomous vehicles.