From Noisy News Sentiment Scores to Interpretable Temporal Dynamics: A Bayesian State-Space Model

Technical Summary: From Noisy News Sentiment Scores to Interpretable Temporal Dynamics

Problem Statement
Text-based sentiment indicators are widely utilized to monitor public and market mood; however, weekly aggregated sentiment series are inherently noisy. A primary source of this noise is the temporal and categorical variability in the volume of relevant news. Some weekly averages rely on hundreds of articles, while others are based on very few. Existing approaches often fail to explicitly account for these fluctuations in data availability when measuring uncertainty. Consequently, standard aggregation methods conflate genuine shifts in sentiment with changes in data reliability. This issue is critical because using AI-generated variables as ground truth in econometric analysis without accounting for measurement error can lead to biased estimates and invalid inference. Furthermore, recent literature emphasizes that uncertainty quantification is essential for making AI signals reliable for financial decision-making.

Methodology
The paper proposes a hierarchical Bayesian state-space model designed to transform aggregated news sentiment into a smoothed time series with principled uncertainty quantification.

• Data Processing: News articles are collected and scored using pretrained NLP models. FinBERT is used to generate sentiment scores ($S_i \in [-1, 1]$), and a zero-shot classification model (bart-large-mnli) assigns relevance scores ($C_{ij}$) to predefined thematic categories. Articles with relevance scores below a threshold of 0.25 are filtered out.
• Aggregation: For each time window $t$ and category $j$, an aggregated sentiment score ($y_{tj}$) is calculated as a weighted average of article-level scores, where weights are the category relevance scores.
• Latent State Dynamics: The model posits an unobserved latent sentiment process ($x_{tj}$) that evolves over time. This process is modeled as an autoregressive process of order one (AR(1)) around a long-run mean ($\mu_j$), governed by a persistence parameter ($\theta_j$) and a latent innovation scale ($\sigma_{\eta j}$).
• Observation Model and Effective Weight: The observed sentiment $y_{tj}$ is treated as a noisy measurement of the latent state $x_{tj}$. The core innovation lies in the observation noise term, which is scaled by an "effective information weight" ($n_{tj}$).
  • $n_{tj}$ is defined as the sum of category relevance scores for all articles in a given window.
  • The observation variance is modeled as $\sigma_j^2 / n_{tj}$.
  • Mechanism: When $n_{tj}$ is high (high coverage), the observation noise is small, and the latent state is anchored strongly to the observed data. When $n_{tj}$ is low (sparse coverage), uncertainty increases, and the model relies more heavily on the latent temporal dynamics to infer the state.
• Hierarchical Structure: To improve robustness, particularly for categories with sparse data, the model employs hierarchical priors. This allows for category-specific parameters (persistence, innovation, and observation noise) while sharing information across categories through hyperpriors.
• Inference: Posterior distributions are estimated using Markov Chain Monte Carlo (MCMC) methods implemented in JAGS.

Key Contributions

1. Information-Density Scaled Noise: The authors introduce a Bayesian state-space model where observation noise is explicitly scaled by an observable, data-driven measure of information density ($n_{tj}$). This directly links data availability to measurement uncertainty, avoiding the need to infer time-varying noise through generic latent volatility.
2. Hierarchical Robustness: The framework utilizes a hierarchical structure to share information across multiple thematic categories. This approach improves estimation stability for categories with fewer messages while still allowing for category-specific behaviors.
3. Uncertainty-Aware Framework: The model provides full posterior distributions for latent states and parameters, enabling the reporting of credible intervals. This facilitates a principled approach to monitoring sentiment dynamics where uncertainty is quantified rather than ignored.

Results
Empirical analysis was conducted on a dataset of approximately 9,918 news articles spanning late 2023 to late 2025, categorized into six thematic groups.

• Parameter Inference: The posterior analysis reveals that the persistence parameter ($\theta_j$) and the latent innovation scale ($\sigma_{\eta j}$) are broadly similar across categories. The differences between a fully pooled model and the hierarchical model are relatively small for these parameters, suggesting that temporal persistence and short-term volatility are shared features across categories.
• Observation Noise Heterogeneity: In contrast, the observation noise parameter ($\sigma_j$) exhibits significant variation across categories. This indicates that the reliability of the aggregated sentiment as a proxy for the latent state varies by topic, likely due to differences in category ambiguity or news content variability.
• Model Validation: A comparison between a heteroscedastic model (where variance scales with $1/n_{tj}$) and a homoscedastic model (constant variance) confirms the core modeling assumption. The heteroscedastic specification demonstrates that posterior uncertainty (interval width) increases mechanically as information density decreases, a relationship that is statistically significant and stronger than in the homoscedastic variant.
• Latent vs. Observed Dynamics: The latent state estimates ($x_{tj}$) provide a smoother trajectory than the raw aggregated scores ($y_{tj}$). The model effectively filters out short-term fluctuations, particularly during periods of sparse coverage, distinguishing signal from measurement noise.

Significance and Claims
The paper positions this framework primarily as a tool for descriptive monitoring rather than forecasting. The authors claim that by explicitly modeling measurement reliability based on information availability, the framework offers a more interpretable and stable representation of sentiment dynamics.

The significance of the work lies in its ability to:

• Separate true sentiment changes from artifacts caused by varying data volume.
• Provide credible intervals that reflect the actual information density of the underlying text sources.
• Offer a generalizable approach for any text-based sentiment analysis where information availability fluctuates over time (e.g., social media, messaging platforms, or news feeds).

The authors maintain a modest scope, acknowledging limitations such as the arbitrary nature of the relevance threshold, the reliance on specific NLP models, and the simplified AR(1) dynamics chosen for parsimony. They do not claim superior forecasting accuracy but emphasize the value of uncertainty quantification for reliable downstream analysis and decision-making.