Impact of LLMs news Sentiment Analysis on Stock Price Movement Prediction

This paper evaluates the impact of LLM-based news sentiment analysis on stock price prediction, demonstrating that DeBERTa outperforms other models and that an ensemble approach achieves 80% accuracy, while sentiment features provide modest improvements to various time-series forecasting architectures.

The Gist

Imagine you are trying to predict the weather for tomorrow. You could just look at the thermometer (the stock price history), but a truly smart forecaster also reads the news, checks the social media mood, and listens to what the locals are saying.

This paper is essentially a report card on how well AI "news readers" help us predict the future of stock prices. The authors wanted to answer a simple question: Does reading financial news with advanced AI actually help us guess if a stock will go up or down, and which AI reader is the best?

Here is the breakdown of their experiment and findings, using some everyday analogies.

1. The Setup: The Three "News Readers"

The researchers tested three different types of Large Language Models (LLMs)—think of these as three different AI journalists with different training backgrounds:

• FinBERT: A specialist. This AI was trained specifically on financial textbooks and reports. It's like a Wall Street veteran who knows all the jargon.
• RoBERTa: A generalist. This AI is smart and reads everything, but isn't specialized in finance. It's like a well-read librarian.
• DeBERTa: Another generalist, but with a slightly different "brain architecture" that allows it to understand context very deeply. It's like a detective who pays extra attention to the nuance of a sentence.

The Finding: Surprisingly, the "detective" (DeBERTa) was the best at reading the news, correctly guessing the mood (positive, negative, or neutral) about 75% of the time. The specialist (FinBERT) was second, and the generalist librarian (RoBERTa) was third.

2. The "Crowd Wisdom" Strategy

The authors realized that no single reader is perfect. Sometimes the specialist misses a joke; sometimes the detective over-analyzes a simple statement.
So, they tried a Team Approach. They took the opinions of all three AIs and fed them into a "Team Captain" (an ensemble model using an algorithm called SVM).

The Result: The team was smarter than any individual member. By combining their votes, the accuracy jumped to 80%. It's like asking three different experts for advice and then having a fourth person make the final decision based on all their input.

3. The "Stock Predictors"

Once the AI readers summarized the news, the team had to feed that information into Stock Predictors. These are the models that actually guess the stock price. They tested four different types of predictors:

• LSTM: The old reliable. It's like a seasoned trader who looks at a long list of past prices.
• PatchTST & TimesNet: The modern tech wizards. These use advanced "patching" and "time-warping" techniques to find patterns in data that humans can't see.
• tPatchGNN: The networker. It looks at how different stocks are connected to each other, like a social network for companies.

4. The Big Reveal: Does News Help?

Here is the most interesting part. The researchers asked: Does adding the "News Mood" actually make the Stock Predictors better?

• For the "Old Reliable" (LSTM): Yes, but only a little bit. Adding news sentiment helped it make slightly better guesses about whether a stock would go up or down.
• For the "Modern Tech Wizards" (PatchTST & TimesNet): Huge improvement! These models were like sports cars that were running on regular gas. When you added the "News Sentiment" fuel, they sped up significantly. They predicted price changes much more accurately.
• For the "Networker" (tPatchGNN): It got a small boost in accuracy for guessing up/down trends.

The Analogy: Imagine you are driving a car (predicting stocks).

• The LSTM is a car with a good GPS. Adding a radio news feed (sentiment) helps you avoid one or two traffic jams.
• The PatchTST/TimesNet are self-driving cars. Without the news, they are driving blind in fog. Turning on the "News Sentiment" radar clears the fog, and suddenly they can drive much faster and safer.

5. How They Did It (The Lab)

• The Data: They looked at 5 famous companies (Apple, Tesla, Microsoft, etc.) over a few years. They collected over 96,000 news articles and matched them to daily stock prices.
• The Aggregation: Since 50 news articles might come out on one day, they had to summarize them into one "Daily Mood Score." They did this by counting the articles, adding up the scores, and taking a "majority vote" (e.g., if 30 are positive and 20 are negative, the day is positive).
• The Test: They ran the experiment 10 times with different random settings to make sure the results weren't just luck.

The Bottom Line

This paper proves that AI is getting really good at reading financial news, and when you combine the best AI readers into a team, they are even better.

Most importantly, news matters. It's not just noise. When you feed this "mood data" into modern, advanced AI stock predictors, they become significantly more accurate. It's like giving a blindfolded runner a pair of glasses; they can finally see the finish line clearly.

In short: If you want to predict the stock market, don't just look at the numbers. Listen to the news, use the smartest AI readers available, and combine them with modern prediction models for the best results.

Technical Summary

Here is a detailed technical summary of the paper "Impact of LLMs News Sentiment Analysis on Stock Price Movement Prediction," accepted at the ICLR 2026 Workshop on Advances in Financial AI (AFA).

1. Problem Statement

Stock price prediction is a critical yet challenging task due to the influence of diverse economic, technical, and political factors. While previous research has separately addressed sentiment analysis (using NLP) and stock movement prediction (using time-series models), there is a lack of comprehensive studies that:

• Compare different Large Language Model (LLM) architectures specifically for financial sentiment.
• Analyze how different sentiment representations (e.g., probability scores vs. discrete labels) and aggregation methods affect downstream prediction.
• Evaluate the fusion of sentiment features with state-of-the-art (SOTA) time-series architectures, such as patch-based transformers and temporal graph neural networks, rather than just traditional models.

2. Methodology

Datasets

• Stock Data: Daily data for five major companies (MSFT, AMZN, AAPL, NFLX, TSLA) from March 10, 2022, to April 2, 2025, sourced from Yahoo Finance.
• News Data: Over 96,000 news articles collected via the AlphaVantage API, mapped to trading days.
• Sentiment Training Data: The SEntFiN 1.0 dataset (10,700+ labeled headlines) was used to train ensemble sentiment models. Note: SEntFiN lacks temporal/stock linkage, so it was used only for sentiment model training, not the final stock prediction pipeline.

Sentiment Analysis Models

The authors evaluated three Transformer-based LLMs:

1. FinBERT: A domain-adapted BERT variant finetuned on financial corpora.
2. RoBERTa: A general-purpose optimized BERT variant.
3. DeBERTa: A general-purpose optimized BERT variant.

• Output: These models generate sentiment classes (Negative, Neutral, Positive) and confidence scores.
• Ensemble Models: Three traditional ML classifiers (Logistic Regression, Random Forest, SVM) were trained on the combined outputs of the three LLMs to create an ensemble sentiment predictor.

Sentiment Aggregation

Since multiple news items occur daily, the authors aggregated article-level sentiments into daily features using:

• Sum: Total sentiment intensity.
• Min/Max: Most pessimistic/optimistic signals.
• Majority Vote: The prevailing sentiment class.
• Count: Number of articles per sentiment label.

Stock Prediction Architectures

The study tested four distinct time-series forecasting families:

1. LSTM: A baseline sequential model.
2. PatchTST: A transformer-based model using patching mechanisms for temporal segmentation.
3. TimesNet: A model employing 2D temporal variation modeling.
4. tPatchGNN: A model integrating temporal patching with Graph Neural Networks to capture asset dependencies.

Experimental Setup

• Tasks:
  • Classification: Predicting binary trend movement (Up vs. Down).
  • Regression: Predicting the price factor (ratio of next-day price to current price).
• Protocol: Temporal splitting (70% train, 10% validation, 20% test) to prevent look-ahead bias. Rolling windows of 30 days were used.
• Evaluation:
  • Classification: F1-Score, AUC, Precision, Recall.
  • Regression: MAE (Mean Absolute Error), RSE (Residual Sum of Squares).

3. Key Results

Sentiment Model Performance

• DeBERTa outperformed both FinBERT and RoBERTa on the SEntFiN dataset, achieving 75.2% accuracy.
• Ensemble Success: Combining the three models via an SVM ensemble classifier yielded the highest performance, reaching ~79.1% accuracy and an F1-score of 0.793.
• Complementarity: Venn diagram analysis confirmed that the models predict different subsets of news correctly, justifying the ensemble approach.

Impact on Stock Classification (Up/Down)

• Baseline: Models without sentiment (NS) generally performed poorly (e.g., LSTM AUC ~0.545).
• Improvements: Adding sentiment features improved performance for LSTM, PatchTST, and tPatchGNN classifiers.
  • LSTM + FinBERT achieved the highest AUC (0.562).
  • TimesNet + DeBERTa achieved the highest F1 (0.540).
• Ablation Study: Removing specific sentiment features (like "count" or "sum") degraded performance, confirming that multiple aggregation types are necessary. The "majority vote" feature had the least impact.

Impact on Stock Regression (Price Factor)

• Architecture Sensitivity:
  • PatchTST and TimesNet showed significant gains when sentiment features were added.
    • PatchTST: MAE reduced from 0.391 (NS) to 0.205 (SVM ensemble).
    • TimesNet: MAE reduced from 0.483 (NS) to 0.254 (RoBERTa).
  • LSTM and tPatchGNN showed minimal sensitivity to sentiment; in some cases, they performed slightly better without sentiment features.

4. Key Contributions

1. Comprehensive LLM Comparison: A direct comparison of DeBERTa, RoBERTa, and FinBERT for financial sentiment, identifying DeBERTa as the superior standalone model.
2. Ensemble Validation: Demonstration that an SVM-based ensemble of these LLMs significantly boosts sentiment prediction accuracy (~80%).
3. Architecture-Specific Insights: The study reveals that the benefit of sentiment analysis is not uniform across all time-series architectures.
   • Transformer-based models (PatchTST, TimesNet) benefit substantially from sentiment integration in regression tasks.
   • GNN-based models (tPatchGNN) and LSTMs show mixed or negligible benefits, suggesting that their internal temporal/relational modeling may already capture relevant market dynamics or that they are less sensitive to external text features in this specific setup.
4. Aggregation Analysis: Identification of which sentiment aggregation methods (sum, count, min/max) are most critical for different prediction tasks.

5. Significance

This paper bridges the gap between NLP and quantitative finance by moving beyond simple "sentiment helps/hurts" conclusions. It provides a nuanced view that:

• Model Selection Matters: The choice of LLM (DeBERTa > FinBERT) and the prediction architecture (PatchTST/TimesNet vs. LSTM) dictates whether sentiment analysis is beneficial.
• Ensemble Strategy: Combining multiple LLMs via traditional ML (SVM) is a robust strategy for generating high-quality sentiment features.
• Practical Application: For practitioners, integrating news sentiment is most effective when paired with modern transformer-based time-series models (PatchTST, TimesNet) for regression tasks, rather than relying solely on traditional recurrent networks.

The authors have open-sourced their codebase to ensure reproducibility, facilitating future research in Financial AI.