💬 NLP

AI Application Gives Users Real-Time Feedback on the Level of Peace in the Social Media Videos They Watch

This paper presents an AI application that utilizes large language models to analyze YouTube video transcripts in real time, successfully measuring five social dimensions relevant to peace and providing users with feedback on their media diet, outperforming traditional sentiment analysis and demonstrating the limitations of models trained on written text when applied to spoken language.

Original authors: P. Gilda (Columbia University), P. Dungarwal (Columbia University), A. Thongkham (Columbia University), E. T. Ajayi (St John's University), S. Choudhary (Columbia University), T. M. Terol (Columbia Un

Published 2026-06-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: P. Gilda (Columbia University), P. Dungarwal (Columbia University), A. Thongkham (Columbia University), E. T. Ajayi (St John's University), S. Choudhary (Columbia University), T. M. Terol (Columbia University), C. Lam (Columbia University), J. P. Araujo (Columbia University), M. McFadyen-Mungalln (Columbia University), L. S. Liebovitch (Columbia University), P. T. Coleman (Columbia University), H. West (Columbia University), K. Sieck (Toyota Research Institute), S. Carter (Toyota Research Institute)

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine your daily news diet is like a meal. For decades, we've been told to check the ingredients of our food to see if it's healthy. But today, most of us get our news from social media videos (like YouTube) instead of curated newspapers. The authors of this paper ask: What is the "nutritional value" of the words we hear in these videos? Do they feed us peace, or do they feed us conflict?

They built a digital tool called BAIT (a Chrome extension) that acts like a "peace nutrition label" for videos. As you watch a video, BAIT analyzes the spoken words in real-time and gives you a score on how "peaceful" or "conflict-driven" the content is.

Here is how they built it, explained simply:

1. The Failed Recipe: Trying to Use a "Written" Cookbook

First, the team tried to teach a computer to recognize peace using a massive library of written news articles.

The Analogy: Imagine trying to teach a chef to recognize the taste of a soup by only reading the written recipe cards. The computer learned very well that certain written words (like "diplomacy" or "truce") usually appear in articles about peaceful countries.
The Result: When tested on written news, the computer was a genius, getting about 97% accuracy.
The Problem: When they tried to use this same "written recipe" to analyze spoken video transcripts, it failed miserably. It tried to label almost every video as "peaceful," even when it wasn't.
Why? The authors realized that spoken language is a different animal than written text. A news anchor speaking on camera uses fillers, pauses, emotional tone, and conversational quirks that don't exist in a printed article. The computer was like a chef trying to judge a live cooking show by only reading the menu; it missed the actual flavor.

2. The New Approach: The "Contextual Taste Tester"

Since the "written recipe" didn't work, the team switched tactics. Instead of looking for specific keywords, they asked the computer to understand the vibe and tone of the conversation.

They focused on five specific "flavors" of peace, identified by decades of social science research:

Compassion vs. Contempt: Is the speaker kind or mocking?
News vs. Opinion: Are they reporting facts or just venting feelings?
Promotion vs. Prevention: Are they talking about building something up or just trying to stop something bad?
Creativity vs. Order: Is the conversation open to new ideas, or rigid and controlling?
Nuance vs. Simplification: Are they seeing the gray areas, or just black-and-white extremes?

To measure these, they tested two types of AI:

The "Word Counter" (Sentiment Analysis): This tool just counts happy or sad words. It was like a thermometer that only measures temperature but doesn't know if it's a sunny day or a storm. It performed very poorly (almost random guessing).
The "Contextual Reader" (Large Language Models): These are advanced AIs that can read a whole paragraph and understand the story behind the words. They act like a sophisticated food critic who understands that saying "This policy is a disaster" with a sarcastic tone is different from saying it with genuine anger.

3. The Results: The "Contextual Reader" Wins

The team had a group of human experts (peace researchers) watch 52 videos and rate them on those five "flavors." Then, they compared the human ratings to the AI ratings.

The "Word Counter" AI was barely better than flipping a coin.
The "Contextual Reader" AI (specifically the latest models) got it right about 60% of the time.
The Analogy: This is a huge win. It means the AI is now almost as good as a human expert at understanding the tone of a video, not just the words. It can tell the difference between a heated debate that is still respectful (peaceful) and a calm-sounding speech that is actually full of hidden contempt (unpeaceful).

4. What BAIT Does Now

The final product, BAIT, is a browser extension that sits next to your YouTube video.

For Viewers: It gives you real-time feedback on your "media diet." It helps you realize, "Oh, I've been watching videos that are full of contempt and simplification for the last hour." The goal is to make you self-aware so you might choose to watch something more constructive.
For Creators: It gives video makers feedback on the tone of their own content, helping them understand how their words land with an audience.

The Bottom Line

The paper claims that while we can't perfectly measure peace yet, we have finally built a tool that can listen to spoken videos and understand the emotional tone much better than previous tools. By using advanced AI that understands context (not just keywords), we can now start measuring the "peacefulness" of the videos we watch, with the hope that simply knowing what we are consuming might help us behave more peacefully in real life.

Technical Summary: Real-Time Measurement of "Peace Speech" in Social Media Videos

Problem Statement
The paper addresses the growing reliance of the global population on social media videos (e.g., YouTube, Facebook) as primary news sources, noting that the tone and content of this language play a critical role in initiating or resolving conflicts. While existing tools analyze "Hate Speech" or political bias, there is a lack of mechanisms to measure "Peace Speech"—language that reinforces peaceful social processes. The authors posit that "We become what we behold," suggesting that exposure to specific linguistic tones influences real-world behavior. The core technical challenge is developing a system capable of quantifying these peace-related dimensions in real-time within unstructured, spoken video transcripts, a domain where traditional text-based models have historically failed.

Methodology
The authors developed BAIT, a Chrome extension application, utilizing a two-pronged approach to build its backend analysis engine:

Supervised Machine Learning on News Text (Baseline):
- Data: A dataset of ~700,000 news articles from 18 countries (News on the Web corpus) was tagged with peace levels based on external indices (e.g., Global Peace Index).
- Models: Three neural architectures were trained: a Fully Connected Feed-Forward network, a Convolutional Neural Network (CNN), and a Revised CNN with max-pooling.
- Process: Text was embedded using OpenAI's text-embedding-3-small and classified as "high-peace" or "low-peace."
- Outcome: These models achieved ~97% accuracy on held-out news text and ~70% accuracy on a distinct news dataset. However, they failed to generalize to YouTube transcripts, classifying 95–100% of video content as "high-peace," indicating a fundamental disconnect between written news syntax and transcribed spoken language.
Social Science Dimensions via LLMs (Primary Approach):
- Framework: Recognizing the lack of external peace indices for video transcripts, the authors defined five social dimensions derived from 59 social science studies: compassion-contempt, news-opinion, promotion-prevention, creativity-order, and nuance-simplification.
- Models:
  - Sentiment Analysis (SA): Google's GoEmotions (RoBERTa-based) was tested but found insufficient due to limited context windows, high neutrality baselines, and the "averaging-out" of emotional volatility.
  - Large Language Models (LLMs): The authors pivoted to context-aware LLMs (Gemini 2.5/3 Pro, GPT-4o/5.1). They employed an iterative prompt engineering process, moving from complex news-based prompts to simplified, generalizable prompts optimized for conversational transcripts.
- Ground Truth: A "gold standard" dataset was created by having three human experts (peace researchers and conflict specialists) independently score 52 diverse YouTube videos on the five dimensions using a 1–5 scale. Inter-rater reliability was high ( $r^2 > 0.86$ for three dimensions).

Key Results

Failure of Text-Only Models: Neural networks trained on written news text could not accurately predict peace levels in spoken video transcripts, confirming that the linguistic properties of written and spoken media differ significantly.
Superiority of LLMs: LLMs significantly outperformed word-level sentiment analysis. While SA models showed a weak correlation with human coders ( $r^2 \approx 0.03$ ), LLMs achieved correlations of $r^2 \approx 0.60$ across the five social dimensions.
Model Performance: Gemini 3 Pro Preview demonstrated the highest correlation with human coders across all dimensions, particularly in the challenging "nuance-simplification" category.
Contextual Reasoning: The results indicate that peace assessment requires contextual reasoning (tone, framing, rhetorical structure) rather than raw sentiment or keyword detection.

Key Contributions

BAIT Application: The deployment of a functional Chrome extension that provides real-time feedback to users and content creators regarding the "peacefulness" of their media diet.
Empirical Validation of LLMs: Demonstrating that modern LLMs can effectively scale human-like analysis of complex social dimensions in video transcripts, achieving statistical agreement comparable to human inter-rater reliability.
Dimensional Framework: Operationalizing five specific social science dimensions (compassion, news, promotion, creativity, nuance) as measurable metrics for "Peace Speech."
Open Documentation: The authors publicly document their analysis engine and human testing results, distinguishing their work from other proprietary or opaque content moderation tools.

Significance and Claims
The paper claims that BAIT serves as a foundational analysis engine capable of measuring "Peace Speech" at scale. The authors argue that by making users self-aware of the emotional and social tone of their media consumption, the tool could potentially influence real-world behavior toward more positive social interactions.

The authors maintain a modest stance regarding future implications. They do not claim that the tool provenly changes behavior yet; rather, they state that the ability to measure these dimensions is the necessary first step to test the hypothesis that self-awareness leads to improved behavior. They envision a potential "virtuous cycle" where user awareness drives demand for higher-peace content, incentivizing creators and platforms to produce such content, but they frame this as a future possibility to be investigated through behavioral testing rather than a current reality. The work is presented as a methodological breakthrough in applying AI to peace studies, bridging the gap between social science theory and real-time digital media analysis.