📈 economics

Digital Influence Beyond Reach: Early Consumer Conversation and Subsequent Product Engagement on TikTok

This study demonstrates that on TikTok, high initial reach often correlates with lower product relevance, whereas the semantic quality and direction of early consumer conversations serve as superior predictors of subsequent product engagement compared to traditional volume-based metrics.

Original authors: Nguyen Quang Truc Tran

Published 2026-09-24

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Nguyen Quang Truc Tran

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the crowded world of social media, a common assumption guides how brands measure success: if a post gets a lot of views, likes, and comments, it must be working. This logic treats all attention as equal, assuming that a high number of reactions means people are interested in the product being advertised. However, this view overlooks a crucial distinction. Just because people are looking at a video does not mean they are thinking about the item inside it. They might be admiring the person filming, laughing at a joke, or simply scrolling past without registering the brand. For companies selling complex items like health supplements, where taste, safety, and specific benefits matter, knowing whether a viewer is actually engaging with the product is far more important than knowing how many eyes passed over the screen.

This question of what attention actually means lies at the heart of a new study examining how people talk about a functional nutrition brand called Värna on TikTok. The researchers wanted to understand if the sheer volume of early activity on a post could predict whether that conversation would eventually turn into a serious discussion about the product itself. They focused on the first few hours after a video is posted, a critical window where the nature of the conversation often shifts. By analyzing thousands of comments from Vietnamese users, the team discovered that the size of a creator's audience does not guarantee that the audience will care about the product. Instead, they found that the quality of the early conversation—specifically, whether people were asking about taste, health benefits, or where to buy the item—was a much stronger signal of future interest than the number of likes or views alone.

The study focused on Värna, a brand of nutritional drinks designed for adults in Vietnam to address specific health needs like blood sugar control, bone strength, and recovery. The researchers gathered data from 198 different videos posted by 149 content creators. They collected nearly 2,800 top-level comments and sorted them into categories to see what people were actually saying. Some comments were off-topic, praising the creator's appearance or the video's music. Others were directly relevant to the product, asking questions like "Is this too sweet?" or "Can people with diabetes drink this?" The team separated these into groups such as sensory comments, which described taste or texture, and functional-health comments, which discussed medical suitability.

One of the most striking findings was the relationship between a creator's reach and the type of conversation they sparked. The study confirmed that creators with more followers naturally generated more views, likes, and raw comments. However, the data showed a clear split: as the number of followers increased, the percentage of comments that were actually about the product decreased. In other words, the biggest stars attracted the most attention, but that attention was often directed at the person rather than the drink. The largest creators had the lowest share of product-focused comments, suggesting that a massive audience does not automatically translate into a relevant discussion about the item being sold.

The researchers also tracked how the conversation changed over time. In the very first hour after a video was posted, the total number of comments was high, but many of them were general reactions. As time passed, the total volume of comments dropped, but the nature of the remaining conversation shifted. Within the first three hours, the proportion of comments discussing the product's sensory qualities, such as sweetness or texture, rose significantly. This indicated that while the initial wave of activity might be broad and shallow, the people who stayed engaged were increasingly likely to be thinking about the product itself. The conversation was not just getting quieter; it was getting more specific and useful.

To test whether these early signals could predict future interest, the team used a computer model to look at the first three hours of data and guess what would happen in the next 21 hours. They tried to predict whether a post would generate at least one comment about the sensory experience of the drink later on. When the model looked only at the content of the video or the total number of early likes, it performed poorly. Adding the volume of early comments and their timing improved the prediction significantly. However, the biggest jump in accuracy came when the model was allowed to read the actual words people used. By analyzing the specific questions and descriptions in the early comments, the model could predict future sensory engagement with very high accuracy. This proved that the words people chose in the first few hours contained valuable information that simple counts of likes could never provide.

The study concludes that digital influence is not a single number but a layered phenomenon. A post can be widely seen without being product-relevant, and a post with fewer views can spark a deep, useful conversation about a product's qualities. For advertisers, this means that watching the number of views is not enough. They need to listen to what people are saying. The first three hours after a post goes live act as a diagnostic window. If the early conversation is filled with questions about taste, health, or price, it suggests the product is resonating. If the conversation is mostly about the creator or unrelated topics, the product may be getting lost in the noise.

This research does not suggest that big creators are useless, but it does show that their reach and product relevance are two different things. A large audience guarantees attention, but it does not guarantee that the attention is focused on the right thing. The study suggests that the best way to measure success is not by how loud the crowd is, but by what the crowd is talking about. By focusing on the quality and direction of early conversations, brands can better understand which posts are truly working and which are merely popular. The findings offer a clearer path for measuring the real impact of social media marketing, moving beyond simple counts to a deeper understanding of consumer interest.

Technical Summary: Digital Influence Beyond Reach

Problem Statement
Current social media analytics predominantly rely on aggregate metrics such as views, likes, and raw comment counts to evaluate campaign success. However, these metrics fail to distinguish between general attention and attention directed specifically toward the advertised product. In the context of influencer marketing, particularly for functional nutrition products where consumers must evaluate health benefits and sensory experiences, a high volume of engagement does not necessarily correlate with product relevance or sales effectiveness. Existing literature indicates a gap between the "reach" of a creator and the "relevance" of the conversation, yet few studies proactively analyze how early consumer dialogue evolves into qualified product discussion or whether early semantic signals can predict subsequent product engagement.

Methodology
The study employs a temporally ordered predictive design using a dataset of 198 TikTok posts from 149 creators promoting the Vietnamese functional-nutrition brand "Värna." The analysis utilizes 2,775 top-level comments from 2,388 unique commenters.

Data Collection & Filtering: Data was scraped using Apify workflows targeting specific Värna-related search strings. Posts were screened for brand relevance, and a quality control rule excluded posts with more than 120 comments to prevent right-censoring effects, resulting in a final analytical cohort of 198 posts.
Engagement Reformulation: Comments were decomposed into non-exclusive semantic categories: product-relevant, functional-health, sensory, purchase/action, question/inquiry, and off-topic. This allowed for the calculation of "qualified engagement" shares (e.g., the proportion of comments discussing sensory attributes like taste or sweetness).
Temporal Windows: The study defines two distinct time windows:
- Observation Window (0–3 hours): Contains predictors including post content features, early interaction volume, timing of interactions, and early semantic signals (product and sensory discussion).
- Outcome Window (3–24 hours): The target variable is binary, indicating whether at least one sensory-product comment occurred in this later period.
Predictive Modeling: A machine learning approach using ExtraTrees classifiers was applied. Models were built cumulatively to test the incremental value of different feature blocks:
- M0: Static content features only.
- M1: Added early volume and timing.
- M2: Added qualified product and sensory semantics.
- M3: Added historical/network-lite variables.
- Evaluation: The models were evaluated using a chronological holdout (30 future test posts) to ensure out-of-time validity. Performance was measured primarily via Precision-Recall Area Under the Curve (PR-AUC) due to class imbalance, alongside ROC-AUC, F1, and Brier scores.

Key Results

Reach-Relevance Divergence: Creator reach (follower count) is positively correlated with views, likes, and raw comment volume ( $\rho > .41$ ) but negatively correlated with the share of product-relevant comments ( $\rho = -.219, p = .002$ ). High-reach creators generate significant attention, but a smaller proportion of that attention is focused on the product itself.
Temporal Evolution: While total interaction volume decays rapidly (dropping from 2.07 comments/post/hour in the first hour to 0.17 in hours 12–24), the composition of the conversation shifts. The share of product-relevant comments increased from 43.9% in the first hour to 52.9% in hours 1–3, and sensory discussion rose from 22.7% to 34.3%. This indicates that product meaning becomes relatively more prominent even as total activity declines.
Predictive Performance:
- Content-only models performed poorly (ROC-AUC: .366).
- Adding early volume and timing significantly improved performance (ROC-AUC: .889).
- Incorporating qualified semantic signals (early product and sensory discussion) yielded the highest performance, achieving a final ROC-AUC of .963 and PR-AUC of .952 on the holdout sample.
- The model achieved a recall of 1.000 with a precision of .545, suggesting its utility as an early detection/screening tool rather than a precise allocation mechanism.
Feature Importance: The most predictive features were the volume of product discussion in the first 3 hours, early sensory involvement, and the persistence of early interaction, rather than static post characteristics or historical network data.

Key Contributions

Redefining Engagement: The study redefines social media engagement as a multi-layered process, distinguishing between general attention and qualified product engagement (functional, sensory, and behavioral).
Reach-Relevance Trade-off: It empirically demonstrates that creator reach does not guarantee product relevance; in fact, higher reach is associated with a lower share of product-focused conversation in this context.
Semantic Predictive Value: The research establishes that the semantic quality of early consumer conversation provides distinct, incremental predictive information about subsequent product experience that cannot be inferred from engagement volume or content alone.
Temporal Analytics: It introduces a framework for analyzing the decoupling of interaction intensity and conversation quality over time, showing that product-relevant discussion can intensify relative to total volume even as the conversation slows down.

Significance and Limitations
The paper claims that digital influence cannot be inferred from reaction volume alone. Instead, the direction and semantic quality of early consumer conversation offer distinct information about subsequent product experience, supporting more adaptive and product-focused advertising analytics. The findings suggest that advertising dashboards should separate scale metrics from qualified engagement metrics, using the first three hours as a diagnostic window to assess product relevance.

The author explicitly notes several limitations:

The study is based on a single brand (Värna) in a specific market (Vietnam), so findings may not generalize to all categories or channels.
The test set was small (30 posts), which may affect the precision of estimates.
The analysis was limited to top-level comments, excluding reply networks and thus missing structural social network metrics like centrality or diffusion pathways.
The semantic labels were rule-based and require blinded manual validation for full reliability.

Consequently, the study positions itself as a proof-of-concept for integrating semantic layers into structural analytics, rather than a definitive, universally applicable model.

Technical Summary: Digital Influence Beyond Reach

More like this