🤖 machine learning

How Much of a 10-K Matters? Aggregation-Dependent Value of Full-Text versus Risk-Factor Sentiment

This paper demonstrates that while full-text 10-K filings yield superior sentiment metrics for sector and portfolio-level predictions of returns and volatility, the narrower Item 1A risk-factor sections outperform at the individual firm level due to the interplay between document volume and available training signal, thereby establishing a supervised lexicon-learning approach as more effective than traditional dictionaries for regulatory disclosure analysis.

Original authors: Sanggyu Sean Choi

Published 2026-07-17

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Sanggyu Sean Choi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the stock market as a giant, noisy ocean where the waves are prices and the currents are investor feelings. For decades, scientists trying to predict these waves have mostly listened to the chatter on the news—short, punchy stories about what companies are doing right now. But there's another, much deeper source of information that has been largely ignored: the massive, dry annual reports companies are legally required to file. Think of these reports as the company's official diary, a thick book filled with every detail of their past year and, crucially, a specific chapter where they must admit all the scary things that could go wrong in the future.

The big question this research tackles is: How do we turn these boring, legal documents into a "mood meter" for the market? Usually, computers just scan these texts with a pre-made list of "good" and "bad" words, like a teacher grading a paper with a fixed rubric. But this paper suggests that a smarter approach is to let the computer learn its own list of words by watching how the market actually reacts. The goal is to see if reading the whole annual report tells us more about future price changes and how shaky those prices might be (volatility) than just reading the specific section where companies list their fears.

The Great Report Card Experiment

In this study, the researchers acted like detectives trying to figure out which part of a company's annual report holds the most secrets about its future. They focused on 1,383 reports from 94 big technology companies (the kind of firms you might find in a tech-focused investment fund) spanning from 2006 to 2023.

They set up a clever game with two main rules:

The Text: They compared reading the entire annual report (the "Full 10-K") against reading just one specific section called "Item 1A," which is dedicated entirely to listing risks and uncertainties.
The Target: They trained their computer models to predict two different things: where the stock price will go next (returns) and how much the price will jitter or shake (volatility).

They tested this in three different ways: looking at the whole technology sector as one big group, looking at a small portfolio of the top 10 companies, and looking at a single company (Nvidia) on its own.

The Twist: Size Matters, But Only Sometimes

The results were a bit like a magic trick where the answer changes depending on how you look at it.

When looking at the big picture (the whole sector or a group of 10 companies):
Reading the entire annual report was the winner. The computer learned better from the massive volume of text in the full document. It was like trying to guess the weather by listening to a whole crowd of people; even if some people are talking about irrelevant things, the sheer number of voices helps you hear the real trend. The full report gave the computer more "training material" to figure out which words actually mattered.

When zooming in on a single company:
The magic flipped! Suddenly, reading just the risk section (Item 1A) became the better choice. Why? Because when you only have one company's history to learn from, the rest of the annual report is full of "boilerplate" stuff—standard legal jargon and financial numbers that don't change much from year to year. It's like trying to guess a single person's mood by reading their entire diary, including pages about what they ate for breakfast every day for ten years. That extra noise drowns out the important clues. But the risk section is short, punchy, and full of the specific worries that actually move the needle for that one company.

What the Words Actually Said

The researchers didn't just look at scores; they peeked at the words the computer decided were important.

For the whole sector: The "good" words were about innovation and health (like "musk" or "torque"), while the "bad" words were about tech struggles.
For the risk sections: These were surprisingly specific. The risk-only model spotted themes the full report missed, like "Supply Chain Risk" (words like "subcontractor" and "dependence") and even specific mentions of "Biotechnology" or "Cybersecurity." It seems the risk section is where companies whisper the specific dangers they are most afraid of, while the rest of the report is just shouting generalities.

The "Old Dictionary" Problem

The study also tested an old, popular method that uses a fixed list of "negative" words (like the Loughran-McDonald dictionary). The result was a bit funny: this old method was consistently wrong in a very predictable way. It was so negative that it actually moved in the opposite direction of the stock price. The researchers explain that this isn't because the dictionary is broken, but because it was designed to be super cautious. Since 10-K reports are legally required to be cautious, the dictionary sees "risk" everywhere, even when the market is feeling fine. This proves that a computer that learns from the data (the supervised approach used here) is much better than one that just uses a static list of words.

The Bottom Line

The paper concludes that there is no single "best" way to read a 10-K report. If you want to understand the mood of the whole tech industry, read the whole book. But if you want to understand the specific fears of one company, skip to the risk chapter. The key takeaway is that the amount of information you need depends entirely on how many companies you are looking at. The more companies you group together, the more text you need to find the signal; the fewer companies you look at, the more you need to cut out the noise and focus on the specific risks.

This research doesn't claim to have solved the stock market, but it provides a new, smarter toolkit for turning legal documents into useful signals, showing us exactly where to look for the truth depending on the size of the crowd we are trying to understand.

Technical Summary: Aggregation-Dependent Value of Full-Text versus Risk-Factor Sentiment in 10-K Filings

Problem Statement

Financial sentiment extraction has historically relied on news text and supervised extraction against return labels alone. Consequently, regulatory filings—specifically the U.S. Securities and Exchange Commission (SEC) Form 10-K—and the target of volatility, which risk disclosure language is arguably best suited to inform, remain comparatively unexplored. While prior work has applied dictionary-based scoring or deep learning to filings, there is a gap in applying supervised lexicon-learning to regulatory text and, crucially, in using volatility as a supervisory target. Furthermore, it remains unclear whether targeted risk disclosure sections (Item 1A) or comprehensive filing text yields more predictive sentiment signals, and how this relationship changes across different levels of aggregation (sector, portfolio, and individual firm).

Methodology

Data and Extraction

The study utilizes 1,383 Form 10-K filings from 94 Nasdaq-100 technology constituents (2006–2023), retrieved from the SEC EDGAR system. The extraction pipeline isolates two document variants for each filing:

Full Filing: The complete HTML text of the 10-K.
Item 1A: The specific risk-factor section, extracted via structural heuristics (removing footers, identifying headings, and locating the section between "Item 1A" and the next major heading).

Despite constituting only 10–15% of the filing length, the Item 1A section retains approximately 57% of the full filing's post-preprocessing word count, indicating a high density of sentiment-relevant vocabulary.

Model Architecture

The paper extends the supervised lexicon-learning framework of Ke et al. [2020], originally designed for news articles, to regulatory filings. The model operates on three core assumptions:

Vocabulary Decomposition: The vocabulary $V$ splits into sentiment-charged words $S$ and neutral words $N$ . A filing's tone is determined solely by the counts of words in $S$ .
Mixture Distribution: Sentiment-charged word counts follow a mixture multinomial distribution governed by an unobserved sentiment score $p_i \in [0, 1]$ , where $p_i=1$ represents pure positive sentiment and $p_i=0$ represents pure negative sentiment.
Sufficiency: The sentiment score $p_i$ is sufficient for the relationship between filing content and the observed label $y_i$ (return or volatility).

Training and Scoring:

Word Scoring: Words are scored based on the fraction of their occurrences in filings with positive labels (Eq. 2). Words are retained as sentiment-charged if their score deviates significantly from 0.5 (governed by thresholds $\alpha_+$ and $\alpha_-$ ) and meets a frequency count $k$ .
Topic Estimation: Since true sentiment scores are unobserved, a proxy $\check{p}_i$ is constructed from the labels. Topic distribution vectors $\hat{O}$ (representing positive and negative sentiment topics) are estimated using a linear projection of relative sentiment-word frequencies against the proxy matrix (Eq. 4).
Sentiment Prediction: For new filings, the sentiment score $\hat{p}$ is the maximizer of a penalized log-likelihood function (Eq. 5), which shrinks the score toward 0.5 for filings with few sentiment words to prevent overconfidence.

Labeling and Aggregation

The model is trained against two distinct targets:

Return: The percentile rank of realized open-to-close log returns.
Volatility: A normalized proxy based on the intra-daily log price range (Corsi [2009]), averaged over a three-day window.

The evaluation is conducted at three levels of aggregation:

Sector: The entire QQQ (Nasdaq-100) technology sector.
Portfolio: A top-10 constituent portfolio.
Firm: Individual firm (specifically Nvidia).

Sector and portfolio-level series are smoothed using a local-level Kalman filter to handle irregular filing arrivals and reduce noise. Firm-level scores are analyzed unfiltered to preserve filing-specific variation.

Key Contributions

Methodological Extension: The first application of a supervised lexicon-learning approach to 10-K filings and their Item 1A risk-factor sections, moving beyond news text.
Target Expansion: The first application of this framework to supervise sentiment extraction against a volatility target, in addition to returns.
Ablation Study: A systematic comparison of full-filing text versus the Item 1A section across three aggregation levels, isolating the predictive value of targeted risk disclosure versus comprehensive text.
Multi-Axis Evaluation: A comprehensive assessment of twelve resulting sentiment metrics (2 document types $\times$ 2 targets $\times$ 3 levels) via classification accuracy, correlation with market outcomes, and qualitative lexical analysis.

Results

Accuracy and Loss

Sector and Portfolio Levels: Full 10-K text outperforms the Item 1A-only section for both return and volatility targets, achieving accuracy margins of 2–5 percentage points.
Individual Firm Level: The trend reverses. The Item 1A section outperforms the full filing by 1 percentage point on return and 6 percentage points on volatility.
Interpretation: This is attributed to an aggregation effect. At the sector/portfolio level, the larger, heterogeneous vocabulary of the full filing provides sufficient material to isolate sentiment-charged terms across many firms. At the single-firm level ( $n=1$ ), the full filing's boilerplate and financial statement language dilute the signal, whereas the narrower, concentrated vocabulary of Item 1A better matches the limited training signal available.

Correlation Analysis

Dictionary Baseline: The Loughran-McDonald (LM) dictionary baseline is consistently and strongly negatively correlated with price across all levels (e.g., $r = -0.91$ at the sector level). This underscores the conservative, risk-averse nature of regulatory language and the limitations of static dictionaries for this domain.
Supervised Scores:
- At the sector level, return- and volatility-supervised scores are negatively correlated ( $r = -0.245$ ).
- At the firm level, return- and volatility-supervised scores are positively correlated ( $r = 0.612$ for full 10-K), contrasting with the sector-level relationship.
- Return-supervised sentiment generally correlates positively with price, while the LM baseline correlates negatively.

Qualitative Lexical Analysis

The models learned distinct themes:

Full 10-K: Captured broad themes like "Innovation and Development" (e.g., musk, torque) and "Financial Health."
Item 1A: Surged specific forward-looking risk narratives often diluted in full text, such as "Supply Chain Risk" (e.g., subcontractor, dependence) at the portfolio level, and "Biotechnology" or explicit "Cybersecurity" themes at the sector level.

Significance and Claims

The paper establishes that the value of text source (full filing vs. risk factor section) is aggregation-dependent. It argues that while full-filing text provides superior sentiment signals for aggregate market analysis due to statistical power, targeted risk sections are superior for individual firm analysis where training data is sparse.

The study claims that supervised lexicon-learning is essential for regulatory disclosure text, as evidenced by the strong negative correlation of the static LM dictionary with price, which the supervised models successfully invert or adapt. These findings and design choices are presented not as a final product, but as the foundational methodology for a subsequent, larger-scale, multi-source system. The authors remain modest, noting limitations such as the bag-of-words representation (ignoring multi-word phrases) and the lack of index-weighting in aggregates, framing the work as a necessary step toward more robust financial sentiment analysis.