SenseAI: A Human-in-the-Loop Dataset for RLHF-Aligned Financial Sentiment Reasoning

This paper introduces SenseAI, a human-in-the-loop dataset of 1,439 financial sentiment examples with reasoning chains and market outcomes designed to align LLMs via RLHF, while revealing predictable error patterns like Latent Reasoning Drift that can be corrected through structured human feedback.

The Gist

Imagine you are trying to teach a very smart, well-read robot how to read the financial news and tell you if a company's stock is going to go up or down.

You might think, "Just give the robot a dictionary of financial words and a list of past news stories." But as this paper explains, that's like trying to teach a chef to cook a perfect steak by only showing them pictures of the finished meal, without ever explaining why the salt was added or how the heat was controlled.

Here is the story of SenseAI, the new tool designed to fix this problem, explained simply.

1. The Problem: The Robot is "Too Polite" and "Too Guessy"

The paper argues that current AI models (like the ones you might chat with) are great at general conversation but terrible at high-stakes financial decisions. Why?

• The "Hedging" Habit: Imagine a robot that is terrified of being wrong. If a news headline says, "Company X had record profits!" the robot doesn't say, "This is great!" Instead, it says, "This is slightly good, maybe, unless the market crashes." It's so afraid of being wrong that it waters down every strong opinion.
• The "Mind Reader" Trap: Sometimes the robot ignores the actual news article and starts guessing based on what it "remembers" about the company from its training data. It's like a student taking a test who ignores the question and just writes down the answer they memorized for last year's exam.
• The "Confidence" Lie: The robot often says, "I am 70% sure," but it turns out it's just as likely to be wrong at 70% as it is at 50%. It doesn't actually know when it's guessing.

2. The Solution: SenseAI (The "Human-in-the-Loop" Tutor)

The authors created SenseAI, which isn't just a list of answers. It's a training manual built on a specific method called "Human-in-the-Loop" (HITL).

Think of it like a driving school for AI:

1. The Lesson: The AI reads a piece of financial news and gives its answer (e.g., "Slightly Bullish").
2. The Instructor: A human financial expert (a "tutor") reads the same news.
3. The Correction: If the AI is wrong or too vague, the tutor doesn't just say "Wrong." They say, "You were too polite. Change 'Slightly Bullish' to 'Bullish' because the profits were huge."
4. The "Why": Crucially, the dataset records the reasoning. It captures the AI's thought process before the correction. This teaches the AI not just what the right answer is, but how to think to get there.

3. The Secret Sauce: The "Goldilocks Zone"

One of the most interesting discoveries in the paper is the "Goldilocks Zone."

Imagine a student taking a test:

• Too Bad: They get everything wrong. (Hard to fix; you have to re-teach the basics).
• Too Good: They get everything right. (No need to teach them).
• Just Right (Goldilocks): They get the direction right (they know the stock is going up), but they get the intensity wrong (they think it's a "maybe" instead of a "definitely").

The paper found that the AI is almost always in this Goldilocks Zone. It's not hallucinating crazy things; it's just being too cautious. This is great news because it means the AI is very close to being perfect. It just needs a little bit of "calibration" to stop being so shy.

4. Why This Matters (The "Real-World" Check)

Most financial datasets are like a history book: they tell you what happened in the past, but they don't tell you if the prediction was actually useful.

SenseAI adds a Reality Check.

• The AI makes a prediction.
• The dataset waits 4 hours.
• It checks the actual stock price.
• If the stock went up as predicted, the AI gets a "Good Job" signal. If it went down, the AI learns it was wrong.

This connects the AI's "thoughts" to real money, which is the only way to know if it's actually learning.

5. The Big Takeaway

The paper concludes that we don't need more data; we need better data.

• Old Way: Give the AI 100,000 simple labels (Positive/Negative).
• New Way (SenseAI): Give the AI 1,439 examples where a human expert explained why the AI was too cautious, showed the AI's thought process, and checked the result against the real stock market.

In a nutshell: SenseAI is a specialized training camp that teaches financial AI to stop being a nervous, over-polite robot and start thinking like a confident, sharp financial analyst. It proves that with the right kind of human feedback, we can fix the specific "personality flaws" of AI in the financial world.

Technical Summary

1. Problem Statement

Current financial Natural Language Processing (NLP) resources, such as the widely used FinancialPhraseBank, suffer from critical structural limitations when applied to modern Large Language Model (LLM) training:

• Lack of Reasoning Context: Existing datasets provide only sentiment labels (e.g., Positive/Negative) without the chain-of-thought (CoT) reasoning used to derive them.
• Absence of RLHF Alignment: They lack the specific structural components required for Reinforcement Learning from Human Feedback (RLHF), such as human preference signals, correction annotations, and reasoning divergence data.
• Static Nature: Most benchmarks are static snapshots that do not capture the dynamic evolution of model errors or allow for longitudinal tracking of model performance.
• High-Stakes Limitations: In financial contexts, misclassification carries direct monetary risk. General-purpose LLMs exhibit systematic failure modes (e.g., over-hedging, hallucination) that are invisible in standard label-only datasets but critical for enterprise deployment.

2. Methodology: The SenseAI Dataset

SenseAI is a proprietary, continuously collected corpus designed specifically for RLHF-aligned training. It employs a Human-in-the-Loop (HITL) engine where AI-generated sentiment interpretations are systematically reviewed and corrected by a human financial expert.

Data Collection & Schema

• Source: Financial news headlines and paragraphs covering 40 US-listed equities across 6 sectors (Tech, Finance, Healthcare, Energy, Consumer, Industrials) and 13 data categories.
• Process: An LLM processes news to generate a headline, sentiment classification (5-tier scale), CoT reasoning, and a confidence score. A human expert then validates this output.
• Schema Dimensions: Each of the 1,439 data points (as of the snapshot) contains 13 fields, including:
  • Input/Output: Ticker, Timestamp, AI Headline, AI Sentiment, AI Reasoning (CoT), AI Confidence.
  • Validation: Human Sentiment (corrected label), Edit Type (severity of correction), HITL Edited flag.
  • Ground Truth: Real-world market price at call and 4 hours later (outcome validation).
  • Metadata: LLM Version, News Paragraph.

Quality Control

• Self-Consistency Protocol: The single expert annotator periodically re-evaluates previous data points without reference to original annotations to maintain a 90% consistency target.
• Edit Taxonomy: Corrections are categorized into four types:
  • Category 0: No correction (Accepted).
  • Category 1: Minor correction (e.g., Slightly Bullish $\to$ Bullish).
  • Category 2: Moderate correction (e.g., Neutral $\to$ Slightly Bullish).
  • Category 3: Complete reversal (e.g., Bullish $\to$ Bearish).

3. Key Contributions

The paper makes five primary contributions to the field of Financial NLP and AI alignment:

1. Introduction of SenseAI: The first continuously collected, HITL-validated financial sentiment corpus enriched with AI reasoning chains, expert correction signals, confidence scores, and real-world market outcome validation.
2. RLHF Structural Alignment: The dataset is architecturally designed to satisfy the three core requirements of RLHF:
   • Human Preference Signals: Explicit flags indicating accepted vs. corrected outputs.
   • Correction Annotations: Pairs of original AI output and human-corrected output with severity metrics.
   • Reasoning Context: Full CoT data allowing models to learn why a correction was made, not just the final label.
3. Novel Empirical Findings: The analysis of the dataset reveals six previously undocumented behaviors of LLMs in financial reasoning (detailed below).
4. Methodological Validation: Demonstrates that high-quality, reasoning-rich data is more valuable for fine-tuning than massive volumes of simple labels.
5. Commercial Framework: Outlines a pathway for enterprise AI agent deployment and dataset licensing.

4. Key Results & Preliminary Findings

Analysis of the 1,439 data points yielded six empirical findings regarding LLM behavior:

• Finding 1: Sentiment Hypersensitivity to Linguistic Qualifiers: LLMs consistently gravitate toward "hedged" intermediate classifications (e.g., Slightly Bullish) even when news contains strong directional signals. 61.3% of AI outputs were "Slightly Bullish," indicating a structural calibration bias.
• Finding 2: Systematic Confidence Over-Hedging: Confidence scores cluster heavily in the 60–69% range (71% of data) regardless of signal clarity. Crucially, confidence scores are not calibrated to accuracy; a 70% confidence score has the same error rate as a 60% score, rendering current threshold-based routing ineffective.
• Finding 3: Latent Reasoning Drift: Models implicitly incorporate global context (historical performance, broader market conditions) into single-document analysis, even when not present in the input text. This "contamination" is only detectable via CoT data.
• Finding 4: The Goldilocks Zone: The error distribution is highly predictable: ~50% of outputs require minor correction (Cat 1), ~49% are accepted, and 0% are complete reversals (Cat 3). This indicates models are "mostly right" but systematically miscalibrated, making them ideal for targeted fine-tuning rather than catastrophic retraining.
• Finding 5: Forward Projection: Models frequently inject assumptions about future events or market conditions not present in the source text, a form of hallucination that undermines grounded analysis.
• Finding 6: Model Version Effects: As model versions improve (e.g., GPT-5.2), the distribution of errors shifts further toward minor corrections (Cat 1) rather than eliminating errors entirely, confirming the persistence of the "Goldilocks Zone" across generations.

5. Significance and Implications

• For Research: SenseAI challenges the assumption that "more data is better," arguing that structured, reasoning-rich data is superior for RLHF. It provides the first empirical evidence of "latent reasoning drift" and "forward projection" in financial NLP.
• For Enterprise Deployment: The findings suggest that current general-purpose LLMs are unsafe for autonomous financial agents due to uncalibrated confidence and systematic hedging. SenseAI provides the necessary training signal to:
  • Improve sentiment accuracy and reasoning coherence.
  • Calibrate confidence scores to actual accuracy (enabling safe automated routing).
  • Ensure regulatory compliance by grounding analysis strictly in provided text.
• Market Position: The dataset is positioned as a high-value, structurally irreplicable asset for AI labs and financial institutions, offering a competitive advantage in building domain-specific financial agents.

Conclusion

SenseAI represents a paradigm shift from static, label-only financial benchmarks to dynamic, reasoning-aware datasets. By capturing the full cognitive context of financial sentiment analysis—including where and why AI reasoning diverges from expert judgment—it provides the foundational data required to align LLMs with the rigorous demands of enterprise financial decision-making. The paper concludes that the "Goldilocks Zone" of correctable model error makes high-quality HITL data both necessary and sufficient for targeted, high-impact model improvement.