💻 computer science

A Computational Model of Message Sensation Value in Short Video Multimodal Features that Predicts Sensory and Behavioral Engagement

Grounded in Message Sensation Value (MSV) theory, this study develops and validates a computational model analyzing multimodal features in short videos, revealing that while higher MSV consistently increases sensory engagement, moderate MSV levels are optimal for driving behavioral engagement.

Original authors: Haoning Xue, Jingwen Zhang, Xiaohui Wang, Diane Dagyong Kim, Yunya Song

Published 2026-04-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Haoning Xue, Jingwen Zhang, Xiaohui Wang, Diane Dagyong Kim, Yunya Song

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a chef trying to create the perfect short video recipe. You have a pantry full of ingredients: bright colors, fast cuts, loud music, smiling faces, and dramatic storylines. In the past, researchers tried to figure out which single ingredient made people hungry (or in this case, made them watch, like, and share). But this paper argues that you can't just look at one spice; you have to taste the whole dish.

Here is the story of this research, broken down into simple concepts and analogies.

1. The Big Question: What Makes a Video "Pop"?

The researchers wanted to understand Message Sensation Value (MSV). Think of MSV as the "spiciness" or "intensity" of a video.

Low MSV: A calm, slow-paced video about gardening.
High MSV: A fast-cut, loud, flashing, high-energy video about a car crash or a magic trick.

The old way of studying this was like asking a human to watch 1,000 videos and manually write down how "spicy" they felt. That takes forever. This team built a robot chef (a computer model) that can taste 1,200 videos in seconds, analyzing 20 different "ingredients" (like brightness, speed of cuts, volume of sound, and facial expressions) to calculate a "Spiciness Score" automatically.

2. The Two Types of "Hungry" Viewers

The study realized that being "engaged" happens in two stages, like eating a meal:

Stage 1: Sensory Engagement (The "Wow" Factor): This is your immediate reaction. Your eyes widen, your heart beats faster, and you feel a rush. It's like seeing a neon sign or hearing a siren. The computer model found that the spicier the video (higher MSV), the bigger the "Wow!" reaction.
Stage 2: Behavioral Engagement (The "Action" Factor): This is what you do after the "Wow." Do you like it? Do you comment? Do you share it with a friend? This is the moment you decide to actually interact with the content.

3. The Big Surprise: The "Goldilocks" Curve

This is the most important finding of the paper, and it uses a classic storybook analogy: The Three Bears.

Too Little Spiciness (Low MSV): The video is boring. It's like eating plain oatmeal. You don't get a "Wow," and you don't bother to like or share it.
Too Much Spiciness (High MSV): The video is chaotic. It's like eating a mouthful of pure wasabi. It shocks your senses (high sensory engagement), but it's so overwhelming that you actually stop interacting. You might feel dizzy or annoyed, so you scroll past without hitting "like."
Just Right (Moderate MSV): This is the Goldilocks Zone. The video is exciting enough to grab your attention but calm enough for you to process it and feel good about sharing it.

The Discovery: The researchers found an Inverted U-Shape.

As videos get spicier, people get more "Wow" (Sensory Engagement goes up).
But as videos get too spicy, people stop clicking "Like" or "Share" (Behavioral Engagement goes down).
The Sweet Spot: The perfect video for getting the most likes and shares has a "Spiciness Score" of about 3.36 (on a scale the researchers created). It's not the wildest video possible; it's the balanced one.

4. Does This Work for Everyone?

The researchers tested their "Robot Chef" on different groups of people:

Age: Did older people like it less? No. The model worked for everyone.
Personality: Did people who love thrill-seeking (high sensation seekers) react differently? Surprisingly, no. Even thrill-seekers preferred the "Just Right" video over the "Too Spicy" one when it came to actually sharing it.

5. The "Secret Sauce" Ingredients

What makes a video hit that "Just Right" score? The computer found that it wasn't just about one thing. The winning combination included:

Pacing: Not too fast, not too slow. (About 12 shots in a 60-second video is the sweet spot).
Faces: Seeing human faces helps, but not if they are screaming or moving too fast.
Brightness & Sound: Needs to be clear and audible, but not blinding or deafening.

Why This Matters (The Takeaway)

If you are a content creator, a marketer, or just someone who wonders why some videos go viral and others don't, here is the lesson:

Don't try to be the loudest, fastest, or most chaotic video on the internet.

If you go too hard, you might grab someone's eye for a split second, but you won't get their heart (or their "Like"). To truly connect with people and get them to share your content, you need to find the balance. You want to be exciting, but not overwhelming. You want to be the "Goldilocks" video—not too cold, not too hot, but just right.

This study gives creators a mathematical way to find that balance, moving away from guessing and toward a science of "perfectly spiced" content.

1. Problem Statement

The contemporary media landscape is dominated by short videos (e.g., TikTok, Instagram Reels), yet existing research on viewer engagement suffers from two main limitations:

Fragmented Analysis: Prior studies focus on individual multimodal features (e.g., brightness, audio tempo) in isolation, failing to capture the complex, integrated nature of how humans process multimodal information collectively.
Theoretical-Methodological Gap: The concept of Message Sensation Value (MSV)—which quantifies how message features attract attention and elicit affective responses—was originally developed for Public Service Announcements (PSAs) using labor-intensive manual annotation. This approach is not scalable for short videos, which differ in format, content diversity, and algorithmic distribution. Furthermore, there is conflicting empirical evidence regarding whether high MSV enhances or distracts from behavioral engagement.

The study aims to bridge this gap by developing a scalable, theory-driven computational model to quantify MSV in short videos and determine its relationship with both sensory engagement (immediate affective attention) and behavioral engagement (actions like likes, comments, shares).

2. Methodology

Data Collection and Sampling

Primary Dataset: 1,200 short videos from Instagram Reels covering eight societal issues (e.g., climate change, conflict).
- Stratified sampling was used to include both "popular" accounts (>1M followers) and "ordinary" accounts (<50k followers).
- Videos were created by 153 unique accounts with an average length of ~40 seconds.
Validation Datasets: Two unseen datasets were used for external validation:
1. Science-related short videos on TikTok ( $N=10,000$ ).
2. Short videos recommended to children across TikTok, Instagram Reels, and YouTube Shorts ( $N=4,492$ ).
- Total Validation Sample: $N = 14,492$ .

Feature Extraction (Multimodal Analysis)

The study extracted 20 multimodal features categorized into static and dynamic visual, face, and audio attributes:

Static Visual: Brightness, Saturation, Warmness, Visual Complexity.
Dynamic Visual: Shot count/second, Shot duration/second.
Face Features (Static/Dynamic): Face age, gender, size, six discrete emotions (disgust, anger, fear, happiness, sadness, surprise), face count/second, and face duration/second.
Audio: Loudness, Tempo, Sound Brightness (Spectral Centroid).
Tools: OpenCV (visual), Google Cloud Video Intelligence API (shots/faces), Face++ (facial attributes), and Librosa (audio).

Human Evaluation (Ground Truth for Sensory Engagement)

Participants: 1,007 U.S. adults recruited via Prolific.
Task: Participants rated 10 random videos each on Perceived Message Sensation Value (PMSV) using a 17-item scale covering emotional arousal, dramatic impact, and novelty.
Role: PMSV served as the ground truth label for "Sensory Engagement" to train the computational model.

Machine Learning Model Development

Approach: Supervised learning to predict PMSV based on the 20 extracted features.
Feature Selection: Five feature sets were tested (including all 20 features, correlation-based subsets, stepwise regression subsets, PLS regression subsets, and an 11-feature "audiovisual-only" set).
Algorithms Tested: Linear Regression, Polynomial Regression, Random Forest, XGBoost, and Support Vector Regression.
Optimal Model: Random Forest Regression using the 11 audiovisual features (excluding specific facial attributes) achieved the best performance.
- Metrics: Mean Squared Error (MSE) = 0.448; Akaike Information Criterion (AIC) = -74.31.
- Key Predictors: Duration of faces, shots per second, and brightness accounted for 52% of feature importance.

Analysis Plan

RQ1 (Sensory): Linear regression to test the correlation between Computational MSV and PMSV.
RQ2 (Stability): Testing model performance across age groups and sensation-seeking preferences.
RQ3 (Behavioral): Negative binomial regression to model the relationship between Computational MSV and behavioral engagement (views + likes + comments), testing both linear and polynomial (non-linear) terms.
RQ4 (Generalizability): Applying the polynomial model to the two unseen validation datasets.

3. Key Results

Sensory Engagement (PMSV)

Positive Linear Relationship: Computational MSV is strongly and positively associated with PMSV ( $b = 1.57, p < .001$ ).
Stability: This relationship holds consistently across different age groups (younger vs. older) and sensation-seeking levels (high vs. low). Higher MSV videos are universally perceived as more sensational.

Behavioral Engagement

Inverted U-Shaped Relationship: While MSV increases sensory stimulation linearly, its relationship with behavioral engagement (likes, comments, shares) is non-linear and inverted U-shaped.
- Linear Term: Not significant when adjusted for covariates.
- Polynomial Term: Significant negative coefficient ( $b = -1.08, p < .001$ ), confirming the curve.
Optimal Point: Behavioral engagement is maximized at a Computational MSV of approximately 3.36.
- Low MSV: Under-stimulating; fails to capture attention.
- High MSV: Over-stimulating; likely distracts from content processing, reducing the likelihood of interaction.
- Moderate MSV: Optimizes the balance between capturing attention and allowing cognitive processing for action.

Validation on Unseen Data

The inverted U-shaped relationship was successfully replicated in both the science-related dataset (Optimal MSV $\approx$ 3.27) and the children's content dataset (Optimal MSV $\approx$ 3.17).
This confirms the model's robustness across different platforms, content types, and demographics.

4. Key Contributions

Theoretical Advancement:
- Resolves conflicting theories (AMIE/LC4MP vs. ELM) by demonstrating that MSV has stage-dependent effects: it linearly drives sensory engagement but follows an inverted U-curve for behavioral engagement.
- Highlights the importance of the "middle ground" of MSV, arguing against the binary view of high vs. low sensation.
Methodological Innovation:
- Developed the first scalable, automated computational model for MSV in short videos, moving beyond manual annotation of PSAs.
- Successfully distinguished between Message-level MSV (objective features) and Observer-level PMSV (subjective perception), using the latter to train a model that predicts the former.
Practical Implications:
- Provides actionable guidelines for content creators: Short videos should aim for a moderate MSV (balanced pace, moderate audiovisual intensity) rather than maximizing sensationalism, to maximize user interaction (likes/shares).

5. Significance and Limitations

Significance:
This research provides a robust, data-driven framework for understanding how multimodal features drive engagement in the algorithmic era. It offers a reproducible tool for researchers to analyze video content at scale and helps creators optimize content strategies based on empirical evidence rather than intuition.

Limitations:

Feature Scope: The model relies on low-level features (color, shot changes) and misses high-level semantic features (e.g., narrative structure, specific objects, AI-generated voiceovers).
Causality: The findings are correlational; experimental designs are needed to confirm causal mechanisms.
Cultural Context: The study is limited to English-language videos in the U.S.; cultural differences in sensation perception may alter the optimal MSV levels.
Algorithmic Influence: While engagement metrics reflect algorithmic curation, the study did not explicitly model how MSV interacts with recommendation algorithms.
Tooling: Reliance on proprietary APIs (e.g., Face++, Google Cloud) may limit reproducibility compared to open-source alternatives.