💬 NLP

HiFi-KPI: A Dataset for Hierarchical KPI Extraction from Earnings Filings

This paper introduces HiFi-KPI, a large-scale dataset containing 1.65 million paragraphs and 198,000 hierarchically organized labels linked to iXBRL taxonomies to facilitate the extraction and classification of Key Performance Indicators from earnings filings, alongside a curated Lite subset for rapid evaluation and open-source code.

Original authors: Rasmus Aavang, Giovanni Rizzi, Rasmus Bøggild, Alexandre Iolov, Mike Zhang, Johannes Bjerva

Published 2026-03-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Rasmus Aavang, Giovanni Rizzi, Rasmus Bøggild, Alexandre Iolov, Mike Zhang, Johannes Bjerva

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to read a financial report from a giant corporation like Apple or Tesla. These reports are like massive, dense forests of numbers and text. To make sense of them, computers need a map.

For years, the financial world has used a digital map called iXBRL. It's like a giant filing cabinet where every single number (like "Revenue" or "Profit") is tagged with a specific label. But here's the problem: the labels are way too specific.

Think of it like this:

The Old Way: Instead of a label saying "Fruit," the computer sees "Red Apple, 2023, Washington State, Gala Variety, 150g." If you want to compare "Apples" across different companies, the computer gets confused because one company calls it "Gala Apple" and another calls it "Red Delicious." It's like trying to compare two libraries where one books are sorted by "Author's Middle Name" and the other by "Book Cover Color." It's a mess.
The Result: Computers are great at reading, but they are terrible at understanding the context (like what currency the money is in, or what dates the numbers cover) when the labels are this hyper-specific.

Enter: HiFi-KPI (The "High-Fidelity" Map)

The authors of this paper built a new tool called HiFi-KPI. Think of it as a universal translator and a smart librarian rolled into one.

Here is what they did, broken down simply:

1. The Massive Collection (The Library)

They scraped 1.65 million paragraphs from thousands of official financial reports (like the 10-K and 10-Q forms companies file with the government).

The Analogy: Imagine they read every single page of every financial report filed in the US for the last seven years. That's a lot of reading!

2. The "Smart Hierarchy" (The Organizing System)

The old system had 198,000 different, confusing labels. HiFi-KPI organized these into a hierarchy (a family tree).

The Analogy: Instead of forcing the computer to memorize 198,000 specific names, they taught it to understand the family.
- Grandparent: "Money Made" (Revenue)
- Parent: "Money from Selling Stuff"
- Child: "Money from Renting out a specific building in 2023"
This allows the computer to zoom in or zoom out. If it can't find the exact "2023 Building Rent," it can still confidently say, "Ah, this is just Revenue." This makes the system much smarter and more flexible.

3. The "Context Clues" (The Detective Work)

A number like "$50 million" is useless without context. Is it US Dollars? Euros? Is it for last year or next year?

The Analogy: HiFi-KPI doesn't just grab the number; it grabs the entire scene. It links the "$50 million" to the specific dates (Jan 1 to Dec 31) and the currency (USD). It's like a detective who doesn't just find a clue but also writes down where they found it and when.

4. The "Lite" Version (The Training Wheels)

The full dataset is huge and complex. So, the authors also created HiFi-KPI-Lite.

The Analogy: This is like a "Study Guide" or a "Flashcard set." They asked a human financial expert to pick the 4 most important things to look for: Earnings, EBIT (profit before interest), EPS (earnings per share), and Revenue.
They mapped thousands of confusing computer tags to these 4 simple human concepts. This allows researchers to test new AI models quickly without needing a supercomputer.

What Did They Test? (The Race)

The authors put different types of AI to the test to see who could read these reports best:

The "Old School" AI (Encoder Models): These are like diligent students who have read the textbook many times.
- Result: They were excellent at identifying the labels (getting 90%+ accuracy). They are great at saying, "This sentence is about Revenue."
The "New Genie" AI (Large Language Models like GPT, Qwen, etc.): These are the flashy, creative AIs that can write stories and chat.
- Result: They were okay at the basics but struggled with the details. They often got the dates wrong or mixed up currencies.
- The Problem: If you ask a Genie to extract a specific number with its date and currency, it sometimes guesses. It's like a smart friend who knows the general idea but might get the specific time of your meeting wrong.

Why Does This Matter?

For Investors: Imagine being able to instantly compare the "Revenue" of 500 companies without manually reading 500 PDFs. This tool automates that, helping investors spot trends faster.
For Companies: It helps them check their own reports for errors before they are filed.
For Regulators: It makes it easier to spot if a company is hiding something or tagging things incorrectly.

The Bottom Line

The paper says: "We built a massive, organized library of financial data that teaches computers how to understand the context of numbers, not just the numbers themselves."

While the newest, flashiest AI models (LLMs) are impressive, the paper shows that for this specific, high-stakes job, specialized, fine-tuned models (the "diligent students") are currently more reliable than the "flashy genies." However, this new dataset gives everyone a better foundation to build even smarter tools in the future.

1. Problem Statement

The paper addresses the challenge of automating the extraction of Key Performance Indicators (KPIs) from financial earnings reports filed with the U.S. Securities and Exchange Commission (SEC).

The Data Source: Public companies are mandated to file reports in inline eXtensible Business Reporting Language (iXBRL). While iXBRL provides machine-readable tags, the underlying taxonomies are extremely complex and fine-grained.
The Core Issue: The iXBRL taxonomy contains up to 198,000 unique labels. These labels are often hyper-specific to individual companies (e.g., aapl:EquitySecuritiesFVNIAccumulatedGrossUnrealizedGainBeforeTax), leading to:
- Poor Generalization: Models trained on one company's specific tags struggle to generalize to others.
- Data Sparsity: Many labels appear very infrequently, making supervised learning difficult.
- Annotation Difficulty: Even for experts, mapping text to the correct specific tag is time-consuming and error-prone (existing data shows >34% error rates in some filings).
- Lack of Context: Existing datasets often strip away crucial contextual information (time periods, currencies, numerical values) embedded in the iXBRL structure, which is vital for accurate financial analysis.

2. Methodology

The authors propose a comprehensive framework involving dataset creation, taxonomy unification, and hierarchical aggregation.

A. Dataset Construction (HiFi-KPI)

Source: 42,018 quarterly (10-Q) and 14,389 annual (10-K) reports from 2017 to June 2024.
Extraction: Using beautifulsoup and regular expressions, the authors extracted text spans associated with iXBRL tags, preserving:
- The text snippet.
- The specific iXBRL tag.
- Associated time periods (start/end dates).
- Numerical values and currencies.
Scale: The final dataset contains 1.65 million paragraphs and 4.5 million entities.
Quality Control: A manual audit of 300 random samples identified errors in date, currency, span, and value parsing. Regular expressions were applied to filter "false starts" and malformed snippets, resulting in a final error rate of <1%.

B. Taxonomy Unification and Aggregation

To solve the hyper-specificity problem, the authors unified the two primary iXBRL taxonomies:

Presentation (.pre): Defines the visual layout and hierarchical relationships in reports.
Calculation (.cal): Defines arithmetic relationships (e.g., $A + B = C$ ).

Bottom-Up Aggregation Algorithm:
Instead of using the raw 198k labels, the authors implemented a bottom-up hierarchical aggregation method.

Starting from leaf nodes (specific tags), labels are iteratively mapped to their immediate parents.
This reduces specificity and increases class cardinality, allowing models to be trained at different levels of granularity (e.g., mapping specific "Operating Lease Income" to the broader "Revenues").
The algorithm selects the most common parent across companies to create a Unified Taxonomy.

C. HiFi-KPI-Lite

For rapid prototyping and evaluation, a curated subset of 8,000 paragraphs was created.

Expert Mapping: A senior domain expert mapped specific iXBRL tags to four general financial concepts: Earnings, EBIT, EPS, and Revenues.
Purpose: Serves as a benchmark for structured extraction tasks where the goal is to extract the label, value, currency, and date.

3. Key Contributions

HiFi-KPI Dataset: The first large-scale resource (1.65M paragraphs) that links iXBRL tags to their full structural context (time, currency, value) and provides a unified view of the taxonomy. It achieves 0% untagged entries and a density of 2.77 tags/entry, surpassing previous datasets like FiNER-139.
Unified Taxonomies: The release of cleaned, unified .pre and .cal taxonomies that allow researchers to select specific levels of granularity for their tasks.
Granularity Selection Method: A novel bottom-up algorithm to collapse specific tags into broader, more generalizable categories, addressing the long-tail distribution of financial labels.
Comprehensive Baselines: Evaluation of three distinct tasks:
- Text Classification (predicting the label of an entry).
- Sequence Labeling (token-level classification).
- Structured Extraction using Large Language Models (LLMs).

4. Experimental Results

Text Classification & Sequence Labeling

Models: Tested BERT-base-uncased, FLANG-BERT (financial domain), and EmbeddingGemma-300M.
Findings:
- Granularity Matters: Coarser-grained labels (via bottom-up aggregation) significantly improved Macro-F1 scores by reducing label sparsity.
- Taxonomy Performance: The Presentation (.pre) taxonomy generally outperformed the Calculation (.cal) taxonomy, likely due to its deeper hierarchy and better distribution of nodes.
- Performance: Encoder-based models achieved >0.906 Macro-F1 on classification tasks for well-represented labels. Sequence labeling models performed exceptionally well, with BERT and FLANG-BERT showing near-perfect performance on the top 1,000 frequent tags.

LLM-Based Structured Extraction (HiFi-KPI-Lite)

Models: Evaluated Gemma-3-27B, Qwen3-30B, Mistral-Small-3.2-24B, and DeepSeek-V3.1.
Task: Extracting a JSON object containing the label, value, currency, and date range.
Results:
- Performance: LLMs struggled with full entity extraction, achieving an F1 score of ~0.440 (best performer: Qwen3).
- Error Analysis:
  - Dates: The primary source of error. Models often failed to parse "three months ended" correctly, confusing start and end dates (e.g., Gemma-27B predicted the same date for start and end).
  - Normalization: Models struggled to normalize values (e.g., converting "$61 million" to 61000000.0).
  - Precision vs. Recall: DeepSeek-V3.1 was conservative (high precision, low recall), while Qwen tended to guess labels even with low confidence (high recall, lower precision).
- Comparison: Fine-tuned encoder models significantly outperformed LLMs on label classification, while LLMs showed promise but required improvement for complex structured extraction.

5. Significance and Future Impact

Investor Value: Accurate automated extraction of KPIs allows for real-time analysis of earnings reports, potentially yielding significant short-term returns and preventing investment errors caused by misinterpreted dates or currencies.
Regulatory Efficiency: Provides legal and regulatory bodies with tools to verify compliance more efficiently.
Research Advancement: HiFi-KPI fills a critical gap in Financial NLP by providing a dataset that leverages the full richness of iXBRL structures, moving beyond simple sentiment analysis to complex, structured information extraction.
Open Science: All code, data, and the bottom-up aggregation algorithm are open-sourced, facilitating reproducibility and further research into hierarchical financial NLP.

Conclusion

The paper demonstrates that while the raw iXBRL taxonomy is too granular for direct generalization, hierarchical aggregation combined with context-rich datasets enables high-performance automated extraction. While encoder-based models excel at classification, LLMs still face challenges in precise structured extraction (particularly with dates and normalization), highlighting a clear path for future research in domain-specific fine-tuning and prompt engineering.