DisSim-FinBERT: Text Simplification for Core Message Extraction in Complex Financial Texts

This study introduces DisSim-FinBERT, a novel framework that combines discourse simplification with aspect-based sentiment analysis to improve the extraction of core messages and sentiment accuracy from complex financial documents like FOMC minutes.

The Gist

Imagine you are trying to understand the mood of a very important, very grumpy, and very wordy chef (the Federal Reserve) who runs the world's biggest restaurant (the economy). Every few months, this chef writes a massive, 50-page diary entry about what's happening in the kitchen.

The problem? The chef writes in a very confusing way. One single sentence might talk about the price of flour, the mood of the waiters, and the temperature of the oven all at once. It's a tangled mess of ideas.

If you ask a standard computer program (like FinBERT) to read this diary and tell you if the chef is happy or sad about the "flour prices," the computer often gets confused. It looks at the whole tangled sentence and says, "Well, there's a lot of stuff here, so I'll just guess 'Happy' because that's the most common guess." It misses the specific point.

This paper introduces a new tool called DisSim-FinBERT. Here is how it works, using simple analogies:

1. The "Sentence Surgery" (Discourse Simplification)

Think of the complex financial sentences as a giant, tangled ball of yarn.

• The Old Way: You try to pull on the whole ball to find a specific color of thread. You get frustrated, and you might pull the wrong thread.
• The New Way (DisSim): Before the computer reads the text, a "sentence surgeon" cuts the tangled ball of yarn into neat, separate strands.
  • Strand A: "The price of flour is going up."
  • Strand B: "But the waiters are happy."
  • Strand C: "The oven is too hot."

Now, instead of one confusing sentence, the computer has three clear, simple sentences. It can look at Strand A and say, "Ah, this is specifically about Inflation (flour prices), and the sentiment is Negative."

2. The "Highlighter" (Finding the Core Message)

The paper explains that in these long financial diaries, the most important part of a sentence is often buried in the middle, surrounded by extra details.

• The Analogy: Imagine a student writing an essay. They might write: "Although the sky is blue and the birds are singing, which is nice, the fact is that the test is tomorrow and I am terrified."
• The Old Computer: Reads the whole thing and gets distracted by the blue sky and singing birds.
• The New Computer: Uses a "Highlighter" to strip away the "blue sky" and "singing birds" (the extra details) and focuses only on the core message: "I am terrified about the test."

This ensures the computer doesn't get the wrong idea just because there was some nice weather mentioned in the same sentence.

3. The "Noise-Canceling Headphones" (Smoothing the Data)

Once the computer reads all these simplified sentences, it creates a chart showing how the chef's mood changes over time. However, because the chef only writes every few months, the chart looks like a jagged, bumpy mountain range with huge gaps. It's hard to see the real trend.

The authors tried different ways to smooth out this bumpy line:

• Moving Average: Like averaging the temperature of the last week. It smooths things out too much and hides the fact that it suddenly got freezing cold.
• The Savitzky-Golay Filter (The Winner): Imagine you are drawing a line through a series of dots on a piece of paper. A simple ruler might miss the curve. This special filter is like a flexible ruler that bends perfectly to follow the shape of the dots. It smooths out the "jitter" (noise) but keeps the sharp turns (the real crises) intact.

Why Does This Matter?

In the past, computers analyzing these financial texts were often wrong. They would say the economy was doing "okay" when, in reality, the chef was panicking about inflation.

By using DisSim-FinBERT:

1. It listens better: It separates the different topics (inflation vs. jobs) so it doesn't mix them up.
2. It sees the truth: It focuses on the main point of the sentence, not the fluff.
3. It matches human intuition: When the authors compared their new computer model to human experts, the new model was 10 times better at guessing what the humans were thinking.

In a nutshell: This paper teaches computers how to untangle messy, complicated financial news, cut out the fluff, and focus on the real message. This helps policymakers and investors understand the economy's true mood, especially when things are going wrong, without getting lost in the confusing language of the experts.

Technical Summary

Here is a detailed technical summary of the paper "DisSim-FinBERT: Text Simplification for Core Message Extraction in Complex Financial Texts."

1. Problem Statement

The paper addresses the challenge of performing Aspect-Based Sentiment Analysis (ABSA) on complex, long-form financial texts, specifically the Federal Open Market Committee (FOMC) minutes.

• Complexity of Input: FOMC minutes contain dense, technical language with multiple aspects (e.g., inflation, employment, growth) often embedded within single, complex sentences.
• Model Limitations: Standard pre-trained models like FinBERT struggle with these texts. Due to the bidirectional nature of BERT and the presence of multiple conflicting topics in one sentence, the model often fails to isolate specific aspects.
• Bias and Misclassification: The baseline FinBERT model exhibits a strong bias toward the "economic growth" aspect (classifying ~95% of sentences as growth-related) and fails to accurately capture sentiment nuances for other aspects like inflation or employment. This leads to skewed sentiment predictions that do not align with human expert analysis or real-world economic events.

2. Methodology

The authors propose DisSim-FinBERT, a novel framework that integrates Discourse Simplification (DisSim) as a preprocessing step before aspect selection and sentiment classification.

A. Discourse Simplification (DisSim)

DisSim is a rule-based, discourse-aware sentence-splitting technique that decomposes complex sentences into a semantic hierarchy:

• Process: It recursively applies 35 hand-crafted grammar rules to split complex sentences into canonical structures (minimal propositions).
• Hierarchy Construction: The output is a tree structure where:
  • Level 0: Represents the key statement (nucleus) containing the core message.
  • Level 1 & 2: Represent subordinate clauses, supporting evidence, or contextual details (satellites).
• Strategy: The model extracts only the Level 0 sentences for the aspect selection phase. This isolates the primary topic, reducing the "noise" caused by secondary clauses that might confuse the aspect classifier.
• Note: Full sentences are retained for the final sentiment classification to ensure comprehensive context is not lost.

B. The DisSim-FinBERT Pipeline

1. Input: Complex FOMC sentences.
2. DisSim Filter: Decomposes sentences and extracts Level 0 core messages.
3. Aspect Selection: Calculates cosine similarity between the Level 0 core messages and predefined aspect vectors (Growth, Employment, Inflation). This replaces the direct classification of the full complex sentence.
4. Sentiment Prediction: The original full sentence is passed to FinBERT for sentiment classification, but the aspect label is determined by the simplified core message.
5. Time-Series Smoothing: To align irregular FOMC release dates with continuous economic data, the authors apply a Savitzky-Golay (SG) filter. Unlike Moving Averages or Hodrick-Prescott filters, the SG filter preserves local peaks and troughs (critical for identifying economic crises) while reducing high-frequency noise.

3. Key Contributions

• Novel Framework: Introduction of DisSim-FinBERT, which bridges the gap between discourse simplification and domain-specific sentiment analysis.
• Improved Aspect Identification: The method successfully breaks the "growth bias" of standard FinBERT, allowing for more accurate identification of inflation and employment aspects.
• Optimized Smoothing Technique: The paper empirically demonstrates that the Savitzky-Golay filter is superior to Moving Averages and HP filters for smoothing financial sentiment time series, as it retains the amplitude and shape of critical economic downturns.
• Public Availability: The implementation (DisSim-FinBERT-FOMC) is made publicly available on Hugging Face.

4. Results

The study evaluates the model on a dataset of 1,030 manually labeled FOMC sentences (covering Growth, Employment, and Inflation) and compares it against the baseline FinBERT and human labels.

A. Aspect Selection Performance

• Baseline FinBERT: Classified 981/1,030 sentences as "Growth" (95.2%), with only 16 for Employment and 33 for Inflation.
• DisSim-FinBERT: Achieved a balanced distribution: 842 (Growth), 28 (Employment), and 160 (Inflation).
• Insight: The simplification step allowed the model to correctly identify sentences where the core message was about inflation, even if the full sentence contained growth-related context.

B. Sentiment Prediction Alignment

• Distribution: The sentiment distribution of DisSim-FinBERT (Neutral: 435, Negative: 352, Positive: 243) closely mirrored human-labeled data (Neutral: 527, Positive: 262, Negative: 241).
• Baseline Failure: Standard FinBERT showed a leftward skew, over-predicting negative sentiment (415 negative vs. 276 positive), leading to overly pessimistic economic interpretations.

C. Statistical Metrics (vs. Human Labels)

Metric	FinBERT	DisSim-FinBERT	Improvement
Correlation	0.013	0.156	~12x higher
Mutual Information	1.873	2.021	Higher information retention
Volatility	0.328	0.374	Closer to human volatility (0.445)

D. Time-Series Analysis

• The DisSim-FinBERT model, when smoothed with the SG filter, successfully captured major economic crises (2008 GFC, 2012 Euro Crisis, 2015 Stock Selloff, 2020 Pandemic).
• Correlation with Human Labels: DisSim-FinBERT achieved a Pearson correlation of 0.39 with human labels, compared to only 0.08 for the non-simplified FinBERT.

5. Significance

• Policy and Market Utility: The framework provides policymakers, central banks, and financial analysts with a more reliable tool for extracting actionable insights from central bank communications. It reduces the risk of misinterpreting complex narratives.
• Economic Forecasting: By accurately capturing sentiment shifts during economic downturns, the model offers a more robust indicator for macroeconomic analysis than traditional text mining approaches.
• Generalizability: The approach demonstrates that preprocessing complex domain-specific text with discourse simplification can significantly enhance the performance of large language models (LLMs) without requiring massive retraining or additional computational resources.