💬 NLP

Can News Predict the Market? Limits of Zero-Shot Financial NLP and the Role of Explainable AI

This paper demonstrates that while zero-shot natural language processing models fail to reliably predict short-term stock movements from financial news, their associated explainability frameworks successfully distinguish trustworthy predictions from unreliable ones, advocating for a shift toward transparent, uncertainty-aware decision-support systems in high-stakes financial contexts.

Original authors: Ali M Karaoglu, Shreyank N Gowda

Published 2026-06-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ali M Karaoglu, Shreyank N Gowda

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to guess whether a stock price will go up, down, or stay the same tomorrow. You have a massive library of financial news articles, and you want to use a super-smart AI to read them all and give you the answer.

This paper asks a simple question: Can we just ask a smart AI (without teaching it specifically about stocks) to read the news and predict the market?

The authors, Ali M. Karaoglu and Shreyank N. Gowda, built a system to test this. Here is the story of what they found, explained with some everyday analogies.

1. The Setup: The "Smart Reader" vs. The "Noisy Library"

Think of the stock market as a chaotic room where people are shouting. The news articles are the shouts.

The Problem: The library is messy. Some articles are just repeats of the same story. Some are written in vague, cautious language ("The company might face some challenges"). Some news is old and irrelevant by the time you read it.
The Tool: They used "Zero-Shot" AI models. Imagine a brilliant student who has read the entire internet but has never taken a specific class on "Stock Trading." You hand them a news article and ask, "Will this stock go up?" They have to guess based on general knowledge, not specific training.

2. The Pipeline: How They Tried to Organize the Chaos

Since the library was messy, they built a four-step assembly line to clean it up before the AI made a guess:

Filtering: They threw away articles that didn't actually talk about the specific company (like throwing away a story about Apple when you are asking about Microsoft).
Timing: They realized that news about an earnings report matters today, but news about a lawsuit might matter in two weeks. They gave "fresh" news more weight and "stale" news less weight, like a chef using fresh ingredients and discarding old ones.
Event Matching: They tried to guess what kind of event the news was (e.g., "Is this a product launch or a CEO firing?") and adjusted the timing based on that.
The "Abstention" Rule: This is crucial. If the news was confusing or weak, the AI was allowed to say, "I don't know" (Neutral) instead of forcing a guess. This is like a judge refusing to give a verdict if the evidence is too shaky.

3. The Results: The "Smart Reader" Got Stuck

They tested this system on 480 different scenarios involving 20 US companies. Here is what happened:

The Prediction Score: The system did not get better at predicting the market than just guessing "Neutral" (staying the same) all the time. In fact, it struggled to predict when stocks would go down.
- Analogy: It's like having a weather forecaster who is great at reading clouds but terrible at predicting rain. They can tell you the sky looks gray, but they can't tell you if you need an umbrella.
The "Negative" Problem: The AI was particularly bad at spotting bad news that would make a stock drop. Even though the AI could understand negative words in a sentence, it couldn't connect those words to the stock price actually falling.
The "Zero-Shot" Limit: The smart student (the AI) who hadn't studied stocks specifically couldn't outperform a simple baseline. This suggests that reading the news alone isn't enough to predict short-term stock moves.

4. The Twist: The "Trust Meter" (Explainable AI)

Here is the most interesting part. Even though the AI couldn't predict the future accurately, the system had a special feature: It could tell you how much it trusted its own guess.

They built a "Trust Meter" that looked at:

How many articles supported the guess? (If only one article said it, the meter goes down).
How consistent were the articles? (If they all came from the same newspaper, the meter goes down).
How fragile was the guess? (If removing just one article changed the prediction from "Up" to "Down," the meter goes down).

The Result: The Trust Meter worked perfectly.

When the meter said "High Trust," the prediction was actually correct about 49% of the time (which is slightly better than random guessing).
When the meter said "Low Trust," the prediction was correct only about 26% of the time.

Analogy: Imagine a weather app that says, "I think it will rain, but I'm only 20% sure because my sensors are broken." You might ignore that prediction. But if it says, "I think it will rain, and I'm 90% sure because I have 10 perfect sensors," you take an umbrella. The system couldn't predict the rain perfectly, but it was excellent at telling you when to trust its prediction.

5. The Conclusion: What This Means

The paper concludes that financial news alone is not a crystal ball. You cannot just feed news into a smart AI and expect it to tell you when to buy or sell stocks. The connection between a news headline and a stock price moving the next day is too weak and too noisy.

However, the system is still useful as a Decision Support Tool. Instead of saying, "Buy this stock," it says:

"Based on these 50 articles, the evidence suggests the stock might go up. However, the evidence is weak and comes from only one source, so you should be very careful."

In short: The AI failed at being a "prophet" (predicting the future), but it succeeded at being a "librarian" (organizing the evidence and telling you how reliable that evidence is). The authors suggest we should stop trying to use AI to replace human traders and start using it to help humans understand why a prediction was made and how much they should trust it.

Technical Summary: Can News Predict the Market? Limits of Zero-Shot Financial NLP and the Role of Explainable AI

Problem Statement
The paper addresses the unresolved challenge of whether financial news can reliably predict short-term stock market movements using Large Language Models (LLMs) without domain-specific fine-tuning. While financial news is a primary source for interpreting market sentiment, converting it into reliable predictions is hindered by data noise, duplication across outlets, implicit language, and complex temporal misalignment between news publication and market reaction. Existing approaches often rely on supervised models trained on labeled datasets, which struggle to generalize across changing market regimes. Conversely, zero-shot approaches offer flexibility but risk producing unreliable signals due to prompt sensitivity and a lack of temporal structure. The authors investigate whether a structured, impact-aware zero-shot pipeline can overcome these limitations to provide trustworthy decision support, even if raw predictive accuracy remains low.

Methodology
The authors propose an impact-aware zero-shot prediction pipeline consisting of four stages:

Data Acquisition and Temporal Alignment:
- News is retrieved via the Finnhub API for specific companies and prediction windows (1, 3, 5, 7, 14, and 31 days).
- A dynamic lookback period is calculated to balance recent coverage against stale information.
- Articles are mapped to specific market trading sessions (US Eastern Time), ensuring that articles published after market close are rolled forward to the next valid session to align with market reaction times.
Preprocessing and Relevance Filtering:
- Deduplication: Title-based deduplication retains the oldest or longest version of a story.
- Entity Matching: Articles are filtered to ensure the target company is mentioned in the headline or body.
- Sentence-Level Filtering: A zero-shot Natural Language Inference (NLI) model (DeBERTa) filters out boilerplate, legal disclaimers, and irrelevant sentences based on a relevance hypothesis.
Impact-Aware Article Weighting:
- Recency Weighting: Articles are weighted based on the number of trading days elapsed, using a decay parameter tuned for specific prediction horizons.
- Event Horizon Classification: A zero-shot NLI classifier categorizes articles into nine event families (e.g., earnings, M&A, regulatory actions). Each family is assigned primary and secondary impact horizons based on event-study literature.
- Horizon Weighting: A Gaussian weighting function aligns the article's expected impact timing with the prediction window.
- Final Weight: A composite weight combines recency, horizon alignment, source coverage (logarithmic), and a discount for headline-only articles.
Prediction and Explainability:
- Sentiment Classification: A RoBERTa model performs zero-shot NLI classification with hypotheses for "bullish," "bearish," and "neutral." Unlike standard softmax, the model uses sigmoid scoring to treat labels independently.
- Abstention Mechanism: Predictions are issued only if scores exceed tuned thresholds ( $\tau_{pos}=0.56, \tau_{neg}=0.26$ ), allowing the system to abstain when evidence is weak.
- Explainability Framework: The system generates multi-layered explanations including:
  - Token-level: LIME attributions for influential sentences.
  - Article-level: Contrastive contribution analysis and greedy flip-set analysis (removing articles to test prediction stability).
  - Aggregate-level: Evidence quality flags (e.g., thin evidence, weight concentration, source diversity) and grounded natural language summaries generated by a local LLM.

Key Contributions

Pipeline Development: The creation of an impact-aware zero-shot pipeline that integrates company relevance filtering, recency weighting, event-dependent impact-horizon weighting, and three-way abstention-based prediction.
Explainability Framework: A multi-layered framework linking predictions to token-level attributions, article contributions, robustness checks (flip-set analysis), and grounded natural language rationales.
Comprehensive Evaluation: An evaluation on a 480-case dataset covering 20 US-listed companies across five sectors and six prediction horizons, comparing the proposed zero-shot NLI configuration against domain-adapted (FinBERT, FinGPT) and general-purpose (Llama 3.1, Mistral 7B) baselines.
Empirical Findings on Limits: Evidence demonstrating that while zero-shot NLP struggles with directional accuracy (particularly for negative movements), structured explainability and evidence-quality diagnostics can effectively distinguish between trustworthy and unreliable predictions.

Results

Predictive Performance: No model, including the proposed zero-shot pipeline, exceeded the majority-class baseline (48.4%) in raw accuracy on the holdout set. This indicates that short-horizon stock prediction from news sentiment alone remains highly difficult.
Model Comparison: The proposed RoBERTa-based NLI pipeline achieved the highest Macro-F1 (0.3409) among tested models, outperforming FinBERT, FinGPT, and generative LLMs. This suggests that benchmark sentiment accuracy does not directly translate to pipeline-level stock prediction.
Negative Class Difficulty: All models struggled significantly with predicting negative movements. Argmax-based models identified only 1 of 38 negative cases, while the threshold-tuned RoBERTa identified 9. The authors attribute this to the weak relationship between article-level sentiment and subsequent price direction, as negative returns are often driven by broader market conditions or missed expectations not captured in news text.
Horizon and Sector Variance: Performance peaked at the 3-day horizon. Sector performance varied, with Consumer stocks showing higher accuracy (largely due to neutral predictions) and Healthcare showing the lowest, likely due to the complexity of regulatory and legal factors.
Explainability Value: The explainability layer provided practical utility. Cases rated "HIGH" on evidence quality achieved 49.4% accuracy (slightly above baseline), while "LOW" rated cases dropped to 26.3%. Flip-set analysis revealed that 10.4% of predictions were fragile (changing after removing a single article), while the median required removing 10 articles to change the label.

Significance and Claims
The paper concludes that zero-shot financial NLP should not be viewed as a standalone source for reliable autonomous trading signals, as it fails to consistently outperform simple baselines in directional accuracy. However, the authors argue for a shift in perspective: the primary value of zero-shot financial NLP in high-stakes settings lies in transparent decision support.

By organizing evidence, quantifying temporal alignment, assessing robustness, and providing reliability ratings, the proposed framework allows users to understand when and why a prediction might be trustworthy or unreliable. The study highlights that while the mapping from news sentiment to price dynamics is structurally limited, the ability to expose the underlying evidence and uncertainty through explainable AI offers significant practical value for risk management and investment analysis.