🤖 AI

Noise-Robust Financial Numerical Entity Attribute Tagging

The paper introduces NORA, a noise-robust framework for Financial Numerical Entity (FNE) tagging that utilizes task-aware instance weighting and a novel evaluation method to effectively handle noisy XBRL labels while jointly predicting rich attributes like concept names, reporting times, scales, and accounting signs on a newly constructed large-scale benchmark.

Original authors: Hsin-Min Lu, Chen-Yang Lai, Yi-Jhen Li, Ju-Chun Yen

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hsin-Min Lu, Chen-Yang Lai, Yi-Jhen Li, Ju-Chun Yen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery inside a massive library of financial reports. These reports are filled with numbers, like "$5 million" or "10.2%." But a number by itself is meaningless. Is that $5 million a profit? A debt? Is it money from last year, this year, or next year? Is it a huge sum or a tiny one?

This is the job of Financial Numerical Entity (FNE) understanding: figuring out the true story behind every number in a financial report.

The Problem: The Library is Messy

The authors of this paper, Hsin-Min Lu and colleagues, point out two big problems with how we currently try to solve this mystery:

The "Source Material" is Flawed: Financial companies file their reports electronically (using something called iXBRL). Think of this like a student filling out a worksheet. Sometimes the student makes mistakes—writing the wrong number, mixing up "profit" with "loss," or getting the date wrong. Because computers often learn by reading these filings, they are learning from a source full of errors. It's like trying to learn math from a textbook where half the answers are wrong.
We're Only Asking One Question: Previous studies mostly just asked, "What is this number called?" (e.g., "Is this 'Revenue'?"). But in the real world, you need to know much more: When does this number apply? How big is the unit (millions vs. billions)? Is it a positive or negative number? Ignoring these details is like trying to describe a car by only saying its color, ignoring its speed, model, and whether it's running.

The Solution: NORA (The Smart Detective)

To fix this, the team built a new system called NORA (Noise-Robust Tagging for Rich Financial Numerical Entity Attributes).

1. The "Noise Filter" (Learning to Ignore Bad Clues)
Imagine you are in a room full of people shouting clues to help you solve a puzzle. Some people are shouting the truth, but others are shouting nonsense or lies. If you listen to everyone equally, you'll get confused.
NORA is like a detective who learns to listen to the volume of the voice. It assigns a "trust score" to every piece of information.

If a clue seems reliable, NORA listens closely.
If a clue seems noisy or contradictory, NORA turns down the volume on that clue so it doesn't mess up the learning process.
This allows the system to learn from the massive amount of data available, even though that data contains mistakes.

2. The "Rich Description" (Asking More Questions)
Instead of just asking "What is this?", NORA asks four questions at the same time for every number:

Tag: What is this concept? (e.g., "Revenue")
Time: Is this a snapshot of a specific day (Instant) or a story over a period (Duration)? Is it past, present, or future?
Scale: Is this number in dollars, millions, or billions?
Sign: Is this a positive gain or a negative loss?

By answering all these questions together, the system gets a much clearer picture of the financial story.

The New Library (The Dataset)

The researchers also built a massive new training ground called the NORA Dataset.

Size: It contains 6.6 million examples. To put that in perspective, previous datasets were like a small bookshelf (1.1 million or 79,000 examples). This is a whole library.
Quality: It includes the full context of the reports, not just the number, so the AI can understand the story around the number.
Realism: It keeps the real-world "noise" (the mistakes) so the system can practice being robust, just like a detective practicing with messy evidence.

The Results: Who Won the Game?

The team tested NORA against other smart systems (like Co-teaching, Mixup, and SSR) using two types of tests:

The Messy Test: Using the raw, error-filled data.
The Cleaned Test: Using a special filter (called NPK) that removes the most obvious errors to see how the system performs on "cleaner" data.

The Verdict:

NORA won. It was the best at figuring out the Concept Name (Tag) and the Time Relation (Time).
It was especially good at understanding Time. This makes sense because figuring out if a number is "past" or "future" requires looking at the whole story, and NORA's ability to ignore noisy clues helped it get this right.
It was very competitive on the other tasks (Scale and Sign), even though those were harder for everyone.

A Special Trick: The "Neighbor Check"

To make sure their results were real, the team invented a way to check if the test data was actually clean. They used a method called NPK (Neighborhood Prior-adjusted KNN).
Think of it like this: If you are trying to guess the answer to a math problem, you look at your neighbors' answers. If 9 out of 10 neighbors have the same answer, you probably have the right one. If your answer is totally different from everyone else's, you might be wrong.
NORA used this "neighbor check" to filter out the most confusing examples from the test set, proving that it really was learning the patterns and not just getting lucky with the noise.

Summary

In short, the paper says: "Financial reports are messy, and old AI models get confused by the mistakes. We built a new system, NORA, that learns to ignore the noise while asking detailed questions about every number. We tested it on a huge, new library of data, and it proved to be the smartest detective in the room, especially at understanding the timing and meaning of financial numbers."

Technical Summary: Noise-Robust Tagging for Rich Financial Numerical Entity Attributes (NORA)

Problem Definition
Financial Numerical Entity (FNE) understanding involves recovering the semantic meaning of numerical mentions in financial reports (e.g., 10-Ks and 10-Qs). While existing benchmarks like FiNER-139 and FNXL have advanced the field, they suffer from two critical limitations. First, they rely on labels derived from inline XBRL (iXBRL) filings, which are often manually prepared and inherently noisy, containing errors such as mismatched tags, missing elements, and incorrect signs. Second, prior work largely reduces the task to predicting only the accounting concept name (tag), neglecting other crucial attributes required for full interpretability: reporting-time relations (e.g., instant vs. duration, past vs. current), measurement scales, and accounting signs. These gaps result in models that are brittle to real-world label noise and fail to capture the full semantic richness of financial data.

Methodology
The authors propose NORA (NOise-Robust Tagging for Rich Financial Numerical Entity Attributes), a framework designed to jointly model multiple FNE attributes while mitigating the impact of noisy supervision.

Noise-Robust Training with Instance-Specific Weighting:
NORA treats FNE understanding as a multi-task prediction problem covering four attributes: tag, time relation, scale, and sign. To handle noisy labels, the model employs a task-aware, instance-specific weighting mechanism.
- The model estimates a noise gate ( $g_a$ ) for each instance and each task $a$ using a shared dense layer and a task-specific sigmoid layer applied to the global [CLS] representation.
- This gate, $g_a \in (0, 1)$ , estimates the noise level of the instance. A value near 0 indicates a reliable instance, while a value near 1 suggests noise or ambiguity.
- The loss function is modulated by this gate: $\tilde{L}_a = (1 - g_a) \cdot L_a + \lambda_a \cdot g_a^2$ . The first term downweights the raw loss for noisy instances, while the second term acts as a regularizer to prevent the model from trivially ignoring all data by setting $g_a \approx 1$ .
- Specific loss functions are tailored to each task: Class-Balanced Cross-Entropy for tags (to handle long-tail distributions), standard Cross-Entropy with class weights for time and scale, and Focal Loss for sign (to address the severe class imbalance of negative signs).
Robust Evaluation via NPK Filtering:
Recognizing that real-world test sets also contain noise, the authors introduce the Neighborhood Prior-adjusted KNN (NPK) filtering method for evaluation.
- This method relies on the assumption of neighborhood consistency: correct labels should align with the semantic neighborhood of an instance.
- Using FAISS, the method retrieves $k$ nearest neighbors for each test instance in the semantic embedding space.
- To correct for class imbalance, a prior-adjusted consistency score is calculated by normalizing neighbor counts against global class priors.
- Instances are ranked by this consistency score, and the top-scoring subset (e.g., top 90%) is used as a lower-noise evaluation set to provide a more reliable assessment of model performance.
Dataset Construction:
The authors constructed the NORA Dataset, a large-scale benchmark comprising 6.6 million narrative instances aligned with iXBRL facts from over 110,000 SEC filings. Unlike previous datasets, it includes rich multi-attribute labels (tag, time, scale, sign) and document metadata (company, period, fiscal year) to support time-relation learning.

Key Contributions

Formulation: The paper formulates FNE understanding as a joint multi-attribute prediction task, explicitly modeling concept name, reporting-time relation, measurement scale, and accounting sign simultaneously.
Framework: It introduces NORA, a noise-robust learning framework that utilizes learnable instance-specific weights to attenuate the influence of noisy labels during training, outperforming state-of-the-art noisy-label baselines (Co-teaching, Mixup, SSR, SelfMix).
Evaluation Protocol: It proposes the NPK filtering method, enabling more reliable model evaluation on real-world noisy test sets by identifying high-confidence subsets based on neighborhood consistency.
Resource: It releases the NORA Dataset, a significantly larger and more attribute-rich resource than existing benchmarks (FiNER-139, FNXL), designed to bridge the gap between idealized benchmarks and realistic financial reporting environments.

Results
Experiments conducted on both unfiltered and NPK-filtered test sets demonstrate NORA's effectiveness:

Performance: NORA achieves the best Accuracy, Macro F1, and Weighted F1 scores for Tag and Time prediction tasks. The improvement is particularly pronounced in Time prediction, where NORA exceeds the next-best baseline by approximately 19.8 percentage points in Accuracy and 26.0 percentage points in Macro F1 on the unfiltered set.
Robustness: While NORA is highly competitive on Scale and Sign tasks (where other methods like Mixup or Co-teaching sometimes lead slightly), it consistently outperforms baselines in the most semantically demanding tasks.
Noise Identification: Analysis of the learned noise gates ( $g_a$ ) reveals that instances with high estimated noise levels for Tag and Time tasks exhibit significantly lower prediction consistency. Manual inspection by a Certified Public Accountant (CPA) confirmed that high-gate instances frequently contain label errors, validating the model's ability to identify unreliable supervision.

Significance
The paper claims that NORA demonstrates the value of jointly modeling rich FNE attributes while explicitly accounting for label noise inherent in real-world financial filings. By moving beyond simple concept-name tagging and addressing the noise problem through instance-specific weighting and robust evaluation protocols, NORA provides a more reliable approach for downstream financial applications such as risk analysis, compliance monitoring, and business analysis. The work highlights that ignoring label noise and attribute richness leads to sub-optimal models, whereas a noise-aware, multi-attribute approach yields superior performance in realistic, noisy environments.