Technical Summary: Generating Fearful Images

Problem Statement

This paper investigates a potential "unvirtuous cycle" within the modern information environment, where generative AI (GenAI) models and online visual media interact to amplify negative emotional content. The authors hypothesize that GenAI systems, trained on vast amounts of digital trace data from social media, may exhibit a systemic bias toward producing images that evoke negative emotions—specifically fear—regardless of the emotional tone of the user's text prompt. This bias could perpetuate a feedback loop: content creators produce emotionally charged media for engagement, AI models over-represent these negative emotions in generated content, and this content is subsequently ingested into future training datasets, further skewing the models.

Methodology

Datasets

The study utilizes two primary datasets:

1. EmoSet: A multiclass dataset of 118,102 images annotated with one of eight discrete emotions (Amusement, Awe, Contentment, Excitement, Anger, Disgust, Fear, Sadness) using the Mikels model. This was used for fine-tuning and validating image emotion recognition models.
2. DiffusionDB: A large-scale text-to-image prompt dataset containing 14 million images generated by Stable Diffusion. The authors analyzed a subset of 10,000 prompt-image pairs (the "2m-first-10k" subset) to compare source prompts against generated images.

Emotion Recognition Pipeline

To evaluate emotional bias, the authors developed and validated automated methods for emotion recognition across both modalities:

• Image Emotion Recognition: The authors evaluated three approaches: zero-shot vision-language models (BLIP, CLIP, ALBEF), an auto-captioning pipeline, and fine-tuned image classifiers. They fine-tuned three architectures (Google's Vision Transformer, Microsoft's Swin Transformer, and ConvNeXT) on the EmoSet dataset. The fine-tuned Google Vision Transformer (ViT) achieved the highest macro-average F1 score (0.7343) and was selected as the primary tool for image analysis.
• Text Emotion Recognition: For the text prompts, the authors employed Demux-MEmo, a state-of-the-art model designed for multi-label emotion recognition. They tested configurations pre-trained on SemEval-2018, Paletz, and GoEmotions benchmarks. Demux-MEmo was chosen for its ability to handle complex, multi-label emotional states and resolve contradictory predictions (e.g., distinguishing between co-occurring emotions like fear and sadness).
• Cross-Modality Alignment: To compare text and image labels, the authors mapped the diverse emotion sets from the text models (e.g., GoEmotions' 27 classes) to the Ekman grouping and the EmoSet categories. This mapping collapsed positive emotions into a single "Joy" label to ensure consistency across modalities.

Bias Measurement

The core objective was to measure "unintended emotional bias" by calculating True Positive Rates (TPR) and False Positive Rates (FPR) for each emotion:

• TPR (Recall): The frequency with which an image evokes emotion E given that the prompt contained emotion E.
• FPR: The frequency with which an image evokes emotion E given that the prompt did not contain emotion E.
• Bias Definition: An emotion is considered biased if the model exhibits both a high TPR (strongly replicating the prompt's emotion) and a high FPR (generating the emotion even when not prompted). The authors used bootstrapping with 1,000 resamples to calculate mean TPR/FPR values and 95% confidence intervals.

Key Results

1. Model Validation

The fine-tuned ViT model demonstrated robust performance on the EmoSet dataset, significantly outperforming zero-shot and auto-captioning baselines. Validation on the external Emotion6 dataset showed high agreement with human annotations, with the exception of "anger," where the model performed notably worse. Crucially, tests on random Gaussian noise images showed the model did not exhibit a strong default bias toward any single emotion, suggesting the observed biases in generation are not artifacts of the recognition model itself.

2. Cross-Modality Correlations

When comparing the full set of emotions between text prompts and generated images, correlations were generally weak. Even after aggregating to the six Ekman emotions, correlations remained low (average cross-modality correlation of 0.0327), with "fear" showing the maximum correlation of 0.0609. This indicates that the emotional distribution in generated images does not naturally mirror the distribution in the source prompts.

3. Evidence of Systemic Fear Bias

The analysis of the DiffusionDB subset revealed a distinct bias toward fear:

• Stable Diffusion: Fear exhibited the highest TPR (approx. 60%) and the highest FPR (near 40%). This suggests that if a prompt evokes fear, the image likely will too; conversely, if a prompt does not evoke fear, there is still a significant chance the generated image will.
• Other Emotions: TPRs for sadness, disgust, surprise, and anger were low (<50%), and FPRs for these emotions were also low (<10%), indicating they are rarely generated unless explicitly prompted. "Joy" was the only other emotion preserved across modalities more than 1/3 of the time.

4. Generalizability Across Models

To determine if this bias was specific to Stable Diffusion, the authors tested:

• OpenAI's GPT-Image-1.5: Using 1,642 prompts from DiffusionDB, the GPT model exhibited a nearly identical bias profile, with fear having the highest TPR and FPR.
• Stable Diffusion XL (SDXL): Using the same prompts, SDXL also showed the highest rates for fear, followed by joy. While SDXL showed slightly more separation in FPRs for other emotions compared to the other models, the dominance of fear remained consistent.

Significance and Claims

The paper claims to provide empirical evidence supporting the hypothesis that generative AI models possess a structural propensity to over-represent fear, regardless of the input prompt. The authors argue this is likely a result of training data over-representing fearful content, which is itself a product of the engagement-driven nature of social media.

The significance of these findings lies in the potential for an "unvirtuous cycle":

1. AI systems generate fear-inducing images even when not prompted.
2. These images enter the digital information environment.
3. Given that negative emotions are contagious and drive engagement, this content receives high visibility.
4. This content is subsequently ingested into the training data for the next generation of AI models, reinforcing and amplifying the bias.

The authors conclude that this bias poses risks to the psychological well-being of users (particularly regarding anxiety) and could influence political mobilization and instability. They advocate for a multidisciplinary approach to better align AI emotion recognition with psychological insights and to develop tools that empower users and designers to mitigate these biases.

Limitations

The paper acknowledges several limitations:

• Label Mismatch: The lack of a perfect 1-to-1 mapping between the emotion taxonomies used for text (GoEmotions, Paletz, SemEval) and images (EmoSet/Mikels) necessitated collapsing categories, potentially obscuring nuances.
• Multi-Class vs. Multi-Label: The EmoSet dataset forces a single emotion label per image (multi-class), whereas human emotional experience is often multi-label. This simplification may not fully capture the complexity of emotional evocation.
• Lack of Neutral Class: The image models are trained to identify a dominant emotion even in neutral images, potentially leading to skewed comparisons with text models that can identify low levels of multiple emotions or neutrality.

The authors call for the creation of new, comprehensive image-emotion datasets that utilize multi-label numerical ratings to address these structural limitations in future work.