Multimodal Analysis of State-Funded News Coverage of the Israel-Hamas War on YouTube Shorts

1. Problem Statement

Short-form video platforms (e.g., YouTube Shorts, TikTok) have become dominant channels for political communication and conflict coverage. However, research into how geopolitical events are represented in this specific format remains limited, particularly regarding state-funded news outlets.

• The Gap: Existing studies often focus on user-generated content or long-form news. There is a lack of systematic analysis on how state-sponsored broadcasters use multimodal signals (text, visuals, audio) to frame the Israel–Hamas war in short-form videos.
• The Challenge: Short videos compress complex events into simple narratives, potentially distorting representation while prioritizing emotional reaction. Analyzing thousands of these videos requires a scalable, automated pipeline that can handle both textual sentiment and visual scene classification without relying on computationally prohibitive Large Language Models (LLMs).

2. Methodology

The authors propose a multimodal pipeline that integrates automatic transcription, dependency-based aspect linking, Aspect-Based Sentiment Analysis (ABSA), and Vision-Language Model (VLM) based scene classification.

A. Data Collection

• Source: 2,371 YouTube Shorts collected over 12 months (post-October 7, 2023) from four state-funded/international broadcasters: Al Jazeera, BBC, Deutsche Welle (DW), and TRT World.
• Filtering: Videos were manually selected and scraped. Non-English content (36 videos) was excluded, resulting in a final corpus of 2,335 Shorts.
• Scale: The analysis covers over 94,000 visual frames and 2,335 transcripts.

B. Textual Processing Pipeline

1. Transcription: Used Whisper large-v3 for speech-to-text. A manual check confirmed superior punctuation and segmentation compared to YouTube's auto-captions.
2. Aspect Linking: Transcripts were parsed using a dependency parser (Dozat & Manning, 2017) to extract dependency triples (Aspect, Head, Role).
   • Lexicon: A custom lexicon of 56 surface forms grouped into 10 categories (e.g., Israel, Palestine, Hamas, Zionism, specific politicians like Netanyahu or Sinwar).
   • Output: Extracted 3,252 aspect-sentence pairs across 782 videos.
3. Aspect-Based Sentiment Analysis (ABSA):
   • Model Selection: Several models were fine-tuned on a gold-standard dataset extended with new validated instances: RoBERTa-base, DeBERTa-v3-base/large, and Qwen2.5-7B (LLM).
   • Result: DeBERTa-v3-base achieved the best performance (Macro-F1 = 81.9), outperforming larger transformers and the LLM. This highlights the efficacy of domain-adapted smaller models over generic large models for structured political text.

C. Visual Processing Pipeline

1. Frame Sampling: Uniform sampling at 1 frame per second (FPS) yielded ~94,000 frames.
2. Scene Classification: Used Qwen3-VL (4B), an open-source Vision-Language Model, to classify frames into a custom 7-category taxonomy:
   • Combat/Military Action
   • Destruction/Humanitarian Crisis
   • Political/Diplomatic Events
   • News Media/Interview
   • Public Protest/Demonstration
   • Symbolic/Religious Ritual
   • Other/Unknown
3. Taxonomy Development: The taxonomy was iteratively refined from 14 to 7 categories to minimize overlap and ambiguity (e.g., merging "civilian life under crisis" into "destruction/humanitarian crisis").

3. Key Contributions

1. Novel Multimodal Pipeline: A reproducible framework combining Whisper, dependency parsing, fine-tuned ABSA, and VLM-based scene classification specifically for short-form political news.
2. Efficiency Finding: Demonstrated that smaller, domain-adapted models (DeBERTa-v3-base) outperform large LLMs (Qwen2.5-7B) and specialized large transformers in political ABSA tasks, advocating for resource-efficient approaches in humanities research.
3. Visual Taxonomy for War Coverage: Developed a robust, 7-class semantic scene taxonomy that captures dominant visual themes in conflict reporting with 86.9% accuracy on a held-out test set.
4. Empirical Dataset: Created a large-scale, annotated dataset of 2,300+ state-funded Shorts covering the Israel–Hamas war, bridging the gap between user-generated content analysis and institutional media analysis.

4. Key Results & Findings

A. Sentiment Dynamics (Textual)

• Outlet Polarization:
  • Al Jazeera (AJ) and TRT World: Showed the strongest polarity. They exhibited sustained negative sentiment toward Israel and positive sentiment toward Palestine.
  • BBC and DW: Maintained a more neutral stance, though negative framings were still more frequent than positive ones overall.
• Audience Engagement:
  • For AJ and TRT, videos with positive sentiment (toward Palestine) received higher median view counts.
  • For BBC, negatively framed videos received higher view counts.
• Temporal Trends: Positive sentiment toward Palestine surged in December 2023 (linked to protests). Sentiment toward "Zionism" remained consistently negative, while "Islam" related terms stayed largely neutral.

B. Visual Patterns (Scene Classification)

• Distribution: The most frequent scene type was News Media/Interview (34.7%), followed by Destruction/Humanitarian Crisis (16.4%) and Political/Diplomatic Events (11.8%).
• Outlet Differences:
  • TRT World had the highest proportion of "Destruction/Humanitarian Crisis" imagery (19%) and "Public Protest" scenes (15%).
  • Al Jazeera had the highest volume of "News/Interview" frames.
• Temporal Correlation: Visual spikes aligned with real-world events. For example, "Destruction" peaks in July–September 2024 coincided with the Khan Yunis crisis and polio outbreaks. "Public Protest" spikes aligned with campus encampments (April–May).

C. Multimodal Alignment

• The study found that textual and visual cues are not always overtly aligned. Affective cues often emerge through a subtle interplay of audio (chants), narrative structure, and text overlays rather than explicit text-image contradictions.
• Edge Cases: The model struggled with ambiguous scenes (e.g., distinguishing between political events and religious rituals involving figures like Ebrahim Raisi), leading to a 13.1% error rate in the "Other/Unknown" category.

5. Significance

• Methodological Impact: The paper validates that resource-efficient, task-specific models are superior to generic LLMs for specific humanities research tasks, making large-scale multimodal analysis accessible to researchers without massive compute budgets.
• Insight into Propaganda/Narrative: It reveals how state-funded outlets use short-form video to amplify polarization. While visual scenes often reflect real-world events (e.g., destruction), the sentiment framing varies significantly by outlet, suggesting that the "narrative" is driven more by the textual/audio commentary and selective aspect highlighting than by the raw visual content alone.
• Future Applicability: The pipeline serves as a template for analyzing short-form content on TikTok and Instagram, offering a way to study algorithmic-driven video environments and the evolution of political discourse.

6. Limitations

• Language: The study is limited to English-language content; results may differ for non-English outputs from the same outlets.
• Taxonomy Granularity: The simplified 7-category taxonomy inevitably loses nuance, leading to some misclassifications in the "Other/Unknown" category.
• AI-Generated Content: The study did not explicitly filter for AI-generated imagery, which could skew the authenticity of the visual analysis.