Technical Summary: Investigating Associational Biases in Inter-Model Communication of Large Generative Models

Problem Statement

Social bias in generative AI is often characterized by performance disparities or "associational bias," where models reproduce stereotypical links between concepts (e.g., activities, emotions) and demographic groups (e.g., gender, age) even without explicit demographic prompts. While prior work has extensively studied bias in static model outputs or predictive fairness, there is a critical gap in understanding how these biases evolve when generative models are deployed in interconnected pipelines. Specifically, when the output of one model (e.g., an image-to-text describer) becomes the input for another (e.g., a text-to-image generator), biases may propagate, interact, and amplify through iterative exchanges. This is particularly concerning for human-centred perception tasks like human activity recognition and affect prediction, where such drifts could lead to unequal treatment in sensitive deployments such as wellbeing assessment or safety monitoring.

Methodology

The authors propose a framework to investigate demographic distribution drift and its underlying mechanisms through two complementary pipelines:

1. Inter-Model Communication Pipeline:

   • Mechanism: An iterative feedback loop alternating between a text-to-image generator (Stable Diffusion 3.5 Large) and an image-to-text describer (LLaVA-Next with Mistral-7B).
   • Variants: The study implements two loop types:
     • Text-seeded: Starts with a prompt describing a concept (e.g., "a person doing sports") to generate an image, which is then described, and the description is used to regenerate the image.
     • Image-seeded: Starts with a seed image from a dataset, which is described, and the description is used to generate a new image.
   • Convergence: The loop terminates when the cosine similarity between consecutive descriptions (text-seeded) or images (image-seeded) exceeds a threshold, indicating stability.
   • Datasets: Experiments utilize PHASE (context-rich, full-scene human activity/affect) and RAF-DB (face-centric affect images).

2. Explainability Pipeline:

   • Goal: To determine if demographic drifts arise from systematic reliance on spurious visual cues rather than concept-relevant features.
   • Technique: Adapts Grad-CAM to a vision-language decoder setting. It computes token-conditioned saliency maps for specific admissible label tokens (e.g., the token "happy" or "sports").
   • Region Aggregation: Activations are aggregated over four semantic regions: Face, Hair, Body, and Background. This allows the authors to quantify whether the model relies on semantically relevant cues (e.g., face for emotion) or spurious correlates (e.g., hair or background).

3. Evaluation Metrics:

   • Demographic Drift: Measured using the Stuart–Maxwell test for marginal homogeneity, Cohen's $\kappa$ for agreement, and weighted Jaccard similarity to compare pre-loop and post-loop demographic distributions (age, gender, ethnicity).
   • Prediction Success: The rate at which the model's selected label matches the ground truth, analyzed across demographic groups to assess fairness disparities.
   • Statistical Analysis: Logistic regression is used to model the effects of the looping stage and gender on prediction success.

Key Contributions

The paper makes four primary contributions:

1. Demographic Distribution Drift (C1): Introduces a pipeline to systematically measure how perceived demographic distributions shift through iterative model-to-model knowledge exchange, addressing RQ1 (Do drifts reflect stereotypical associations?).
2. Regional Associations (C2): Proposes an explainability pipeline combining token-conditioned saliency with region-based aggregation to assess if drifts are caused by systematic mechanisms (e.g., reliance on hair/background) rather than noise, addressing RQ2.
3. Prediction Success (C3): Evaluates downstream task performance (emotion/activity prediction) across demographic groups to quantify disparities resulting from inter-model exchange, addressing RQ3.
4. Mitigation Strategies (C4): Outlines potential interventions spanning data-centric (augmentation, targeted collection), training-time (adversarial training, attribute-aware objectives), and deployment-time (prompt engineering, guardrails) strategies.

Key Results

• Demographic Drift: Across both PHASE and RAF-DB, the inter-model loop consistently induces demographic drift.
  • Age: Outputs trend significantly toward younger representations for both activities and emotions.
  • Gender: Outputs trend toward female-presenting representations, particularly for emotions (e.g., "Happiness" and "Anger" in RAF-DB saw female representation increase from ~59% to ~92%). Interestingly, for "Sports" (typically male-stereotyped), the drift was also toward female representations, challenging some existing stereotypes while reinforcing others.
  • Ethnicity: In RAF-DB, there was a reduction in Asian representations and an increase in Black and White representations depending on the emotion.
• Spurious Grounding (RQ2): Explainability analysis reveals that models often rely on spurious visual regions.
  • For emotions, where the face should be the primary cue, models frequently showed high activation on hair and background regions.
  • For activities, while body and background cues are relevant, the reliance on hair (demographically correlated) suggests the model uses appearance as a "shortcut" for concept prediction.
• Prediction Success & Fairness (RQ3):
  • Sports (PHASE): Looping improved overall accuracy but worsened gender disparity, with male subjects having higher success rates post-loop.
  • Happiness: Results were dataset-dependent. In PHASE, looping reduced gender disparity; in RAF-DB, it amplified the disparity (favoring females) while drastically lowering overall accuracy.
  • Anger: No significant effects of looping or gender were found in either dataset.
  • Conclusion: Optimizing for performance (accuracy) via iterative loops does not guarantee equitable outcomes and can sometimes degrade fairness.

Significance and Claims

The paper claims that inter-model communication in human-centred AI systems introduces a new vector for bias propagation. Unlike static bias, these biases evolve and can be amplified through repeated exchanges. The study demonstrates that:

1. Drift is Systematic: Demographic shifts are not random noise but reflect systematic model behaviors, often relying on spurious cues (like hair or background) rather than concept-relevant features.
2. Context Matters: The nature of the dataset (close-up faces vs. full scenes) influences how these biases manifest and propagate.
3. Performance $\neq$ Fairness: Improvements in prediction accuracy after looping can coincide with increased demographic disparity, highlighting the need for explicit fairness constraints.
4. Need for Guardrails: The authors emphasize that "plug-and-play" integration of generative models requires careful safeguards, including continuous monitoring of demographic drift and the use of explainability as a diagnostic tool to detect reliance on biased cues.

The work concludes that addressing these issues requires a holistic approach involving data curation, training interventions, and deployment-time controls to prevent the reinforcement of stereotypes in interconnected AI pipelines.