FinAnchor: Aligned Multi-Model Representations for Financial Prediction

1. Problem Statement

Financial prediction from long-form documents (e.g., earnings call transcripts, regulatory filings, central bank communications) faces two primary challenges:

• Signal Sparsity and Noise: Actionable financial signals are often subtle and buried within extensive boilerplate text, recurring patterns, and noise.
• Representation Instability: Different Large Language Models (LLMs) possess distinct inductive biases. When processing the same financial document, they may attend to different evidence spans, leading to heterogeneous rationales and inconsistent predictions. Simply averaging embeddings from different models is ill-defined because they reside in incompatible feature spaces (different coordinate systems).

The core question addressed is: How can we systematically combine heterogeneous LLM representations to leverage complementary evidence without fine-tuning the base models, thereby achieving robust and stable financial predictions?

2. Methodology: FinAnchor Framework

The authors propose FinAnchor (Financial Anchored Representations), a lightweight framework that aligns and aggregates embeddings from multiple frozen LLMs without fine-tuning the underlying models.

A. Core Components

1. Anchor Selection: One LLM is selected as the "Anchor" (e.g., Gemma-2-9B). Its embedding space serves as the common coordinate system.
2. Linear Alignment: For every other "Source" LLM, a lightweight linear mapping (Ridge Regression) is learned to project its embeddings into the Anchor's space.
   • Training: The mapping matrix $W_m$ is learned only on the training split using the source embeddings ($X^{(m)}$) and anchor embeddings ($X^{(a)}$).
   • Objective: Minimize the reconstruction error with L2 regularization:
     $$W_m = \arg \min_{W} \| \tilde{X}^{(m)}W - \tilde{X}^{(a)} \|_F^2 + \alpha \|W\|_F^2$$
   • Inference: Source embeddings are projected via $W_m$ to obtain aligned representations $\hat{x}^{(m \to a)}$.
3. Aggregation: Once all representations are in the same coordinate space, they are aggregated using a fixed operator (e.g., averaging the anchor representation with the aligned source representations).
4. Lightweight Readout: A small classifier (e.g., a Multi-Layer Perceptron) is trained on top of the aggregated representation to perform the final prediction.

B. Key Design Principles

• No Fine-Tuning: The base LLMs remain frozen, preserving their pre-trained knowledge and reducing computational costs.
• Auditability: The linear alignment maps are transparent and interpretable.
• Time-Based Splitting: To ensure realistic evaluation of out-of-time generalization, datasets are split chronologically rather than randomly.

3. Experimental Setup

• Tasks:
  1. Earnings Surprise Prediction: Binary classification on Conference Calls, 10-Q filings, and Nasdaq News.
  2. Stock Movement Prediction: Binary classification of next-day stock price direction based on earnings call transcripts.
  3. FOMC Stance Classification: Three-way classification (Dovish, Hawkish, Neutral) of Federal Open Market Committee communications.
• Baselines:
  • LLM Zero-shot and Few-shot prompting (Gemma 3).
  • Fine-tuned Long-document models (Longformer, Hierarchical FinBERT).
  • Single-LLM representations (using the same readout architecture as FinAnchor but without multi-model alignment).
• Models Used: Gemma-2-9B (Anchor), Llama-3.1-8B, and Qwen3-8B (Sources).

4. Key Results

FinAnchor consistently outperformed all baselines across all five datasets and tasks.

• Performance Gains:
  • Earnings/Text Tasks: Achieved the highest Accuracy and F1 scores on Conference Calls, 10-Q, and FNSPID Nasdaq News, outperforming the strongest single-LLM baseline (Gemma-2) by clear margins (e.g., +1.7% Accuracy on Conference Calls).
  • Challenging Tasks: Even on difficult tasks like Stock Movement and FOMC Stance (where signals are weaker), FinAnchor achieved top results (e.g., 0.599 Accuracy vs. 0.578 for Llama-3.1).
• Comparison:
  • Outperformed prompt-based methods (Zero/Few-shot) significantly, especially on long documents.
  • Surpassed traditional fine-tuned encoders (Longformer, FinBERT) and single-LLM baselines.

5. Analysis and Interpretability

The paper provides rigorous analysis to prove that gains are driven by systematic signal rather than noise:

• Error Overlap: Different models make different errors (pairwise error overlap is ~0.78–0.84, not 1.0). This non-identical failure pattern allows the alignment to correct specific model weaknesses.
• Decision Transitions: The alignment process specifically converts False Positives to True Negatives (FP $\to$ TN). In financial contexts, this is critical as it reduces spurious "buy" signals on days that actually decline, thereby improving risk control.
• Confidence Shifts: On corrected cases, the FinAnchor assigns significantly higher probability to the ground-truth label compared to the single anchor model, indicating a genuine increase in confidence rather than random drift.
• Case Study (Evidence Reweighting):
  • In a specific earnings call example where Gemma alone predicted a "Positive Return" (False Positive), FinAnchor corrected it to "Negative."
  • Mechanism: The alignment amplified the influence of bearish financial cues (e.g., "margin compression," "inventory buildup") that were present in the text but weakly weighted by the anchor model alone. The linear alignment successfully reweighted these complementary signals.

6. Significance and Contributions

• Novel Framework: Introduces a simple, deployable method to unify heterogeneous LLM representations via linear alignment, avoiding the complexity of fine-tuning or complex ensemble learning.
• Robustness: Demonstrates that combining frozen models via an anchor space yields more stable and accurate financial predictions than relying on a single model, even in noisy, long-document scenarios.
• Practical Utility: The method is computationally efficient (only linear maps and a small readout are trained) and offers interpretability, allowing practitioners to understand why a prediction changed (e.g., via confidence shifts and evidence reweighting).
• Risk Management: By specifically reducing false positives (spurious buy signals), the framework offers tangible value for risk management in financial trading.

7. Limitations

• Scope: Experiments are limited to financial English texts and a specific set of backbone models.
• Hyperparameters: The approach introduces choices regarding alignment strength and aggregation strategies, requiring further sensitivity analysis for broader adoption.
• Generalization: The findings need validation across a wider range of languages and non-financial domains.