🤖 machine learning

A Benchmark of State-Space Models vs. Transformers and BiLSTM-based Models for Historical Newspaper OCR

This paper introduces the first OCR architecture based on State-Space Models (Mamba) for historical newspapers, demonstrating that it achieves competitive accuracy with significantly improved computational efficiency and memory scaling compared to Transformer and BiLSTM-based baselines.

Original authors: Merveilles Agbeti-messan, Thierry Paquet, Clément Chatelain, Pierrick Tranouez, Stéphane Nicolas

Published 2026-04-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Merveilles Agbeti-messan, Thierry Paquet, Clément Chatelain, Pierrick Tranouez, Stéphane Nicolas

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a librarian in a massive, ancient library. Your job is to read millions of old, crumbling newspapers and type out exactly what they say so people can search for them on computers. These newspapers are tricky: the ink is faded, the paper is stained, the fonts are weird (some look like gothic script), and the text is sometimes squished together in messy columns.

This paper is about a new tool the librarians are testing to do this typing job faster and better.

The Problem: The "Over-Thinker" vs. The "Speed Reader"

For the last few years, the best tools for this job have been based on Transformers. Think of a Transformer like a meticulous, over-achieving student.

How it works: When it reads a sentence, it looks at every single word and checks how it relates to every other word in the sentence to understand the context.
The Good: It's incredibly smart and accurate.
The Bad: It gets overwhelmed. If the sentence is short, it's fine. But if the sentence is a whole paragraph (like a whole newspaper column), this student has to do a massive amount of math to connect every word to every other word. It gets slow, uses up all the computer's memory, and takes forever to finish the job.

The authors of this paper asked: "Is there a way to be just as smart, but much faster?"

The New Contender: The "Mamba" (State-Space Models)

Enter the Mamba model. Think of Mamba like a highly efficient, focused speed-reader.

How it works: Instead of looking back at everything it just read to understand the current word, Mamba keeps a running summary in its "mind" (a state). It updates this summary as it reads, keeping only the most important information and forgetting the rest.
The Analogy: Imagine reading a long novel.
- The Transformer stops after every sentence to re-read the whole book from page one to make sure it understands the current sentence.
- The Mamba just remembers the plot so far and moves to the next sentence. It doesn't need to re-read the whole book.

What Did They Do?

The researchers built a new system using this "Mamba" technology and tested it against the old "Transformer" students and some other older methods (like BiLSTMs, which are like a slightly older, slower version of the speed-reader).

They tested these systems on real, messy, historical newspapers from Luxembourg. They didn't just test short words; they tested:

Single lines of text.
Whole paragraphs (which are much longer and harder).

They also tested three different "reading styles" for the Mamba:

Autoregressive: Reading one letter at a time, guessing the next based on the previous ones (like writing a story).
CTC: Looking at the whole image and guessing the letters all at once, then cleaning up the mess.
Non-Autoregressive: A middle-ground approach.

The Results: The Race is On

Here is what happened when they put them to the test:

1. Accuracy (Who got the words right?)

The Good News: All the modern AI models (Transformers, Mamba, etc.) were actually very good. They all got about 98% of the letters right on short lines.
The Bad News: When the text got very long (whole paragraphs) or very messy, the "over-achieving student" (Transformer) started to struggle with the sheer volume of data, while the "speed-reader" (Mamba) stayed steady.

2. Speed and Memory (Who finished first?)

The Big Winner: The Mamba model was the clear winner for long texts.
- It was 2 times faster than the best Transformer model when reading whole paragraphs.
- It used half the computer memory.
- The Analogy: If the Transformer needs a giant warehouse to store its notes while reading a long paragraph, the Mamba only needs a small notebook. This means you can run many Mamba models at the same time on one computer, but you'd need a supercomputer to run the same number of Transformers.

3. The "Off-the-Shelf" Test
They also tested famous, ready-made tools like Google's Gemini and Tesseract.

Result: These tools were okay, but they weren't as good as the custom-trained models. It's like using a generic dictionary vs. a specialist who has studied only 19th-century newspapers for years.

The Takeaway

This paper proves that State-Space Models (Mamba) are a game-changer for digitizing history.

For short notes: The old methods are fine.
For massive archives (millions of pages): The new Mamba models are the future. They are fast, they don't crash your computer, and they are just as accurate as the heavy, slow giants.

In simple terms: The authors found a way to make the computer "read" history without getting a headache from the length of the text. This means libraries can finally digitize their entire collections much faster and cheaper, making our history accessible to everyone.

1. Problem Statement

The digitization of historical newspapers by cultural heritage institutions faces significant challenges due to:

Document Degradation: Issues like bleed-through, blur, and varying print quality.
Complex Layouts: Multi-column structures, advertisements, and lack of standardized line segmentation.
Linguistic Diversity: Multiple languages (e.g., Luxembourgish, French, German) and archaic typography (Fraktur vs. Antiqua).
Scalability Bottlenecks: While Transformer-based models (e.g., TrOCR, DAN) achieve high accuracy, their $O(n^2)$ computational complexity (due to self-attention) and sequential autoregressive decoding make them inefficient for processing long text sequences (paragraphs) and large-scale batch deployment.

The authors aim to investigate State-Space Models (SSMs), specifically the Mamba architecture, as a linear-complexity ( $O(n)$ ) alternative to Transformers for Optical Character Recognition (OCR) in this domain.

2. Methodology

2.1 Proposed Architecture: Mamba-based OCR

The authors introduce the first OCR architecture based on SSMs, featuring a shared visual encoder and three distinct decoding strategies:

Visual Encoder: A shared CNN backbone (5 layers with BatchNorm and Max-Pooling) adapted from the DAN model, extracting 2D feature maps flattened into sequences.
Context Modeling: A Bidirectional Mamba connector processes the visual sequence to capture global context in both directions.
Decoding Variants:
1. Mamba-AR (Autoregressive): Uses unidirectional Mamba layers with causal masking. It predicts tokens sequentially, conditioning on previous tokens (similar to standard Transformers but with linear complexity).
2. Mamba-CTC: Uses unidirectional Mamba layers with Connectionist Temporal Classification loss, allowing parallel decoding without explicit alignment.
3. Mamba-NAR (Non-Autoregressive): Uses static learned queries to predict all positions simultaneously via cross-entropy loss.

2.2 Baselines and Comparisons

The study conducts a systematic benchmark against:

Transformer-based Models:
- DAN: Autoregressive Transformer (strong baseline).
- DANIEL: Autoregressive Transformer using Byte-Pair Encoding (BPE) subword tokenization.
- VAN: CTC-based Transformer with implicit line segmentation.
Recurrent Models: PERO-OCR (CNN-BiLSTM-CTC).
Off-the-Shelf Systems: Tesseract v5, TrOCR (zero-shot and fine-tuned), and Google Gemini.

2.3 Datasets and Evaluation

Source: Bibliothèque nationale du Luxembourg (BnL) historical newspapers.
Data:
- Line-Level: 33,000 lines (Fraktur and Antiqua scripts) with >99.95% verified ground truth.
- Paragraph-Level: 23,939 text regions, with a specific focus on severely degraded paragraphs (1–10 lines, <1000 chars).
Metrics: Character Error Rate (CER), Word Error Rate (WER), Inference Latency, Throughput (images/sec), and Peak GPU Memory usage.

3. Key Contributions

First SSM Application in Document OCR: Introduces Mamba-based architectures for historical document recognition, demonstrating their viability as a scalable alternative to Transformers.
Unified Benchmark: Provides the first controlled comparison of SSMs, Transformers, and BiLSTMs across three decoding paradigms (CTC, AR, NAR) on the same historical dataset.
Dual-Granularity Analysis: Evaluates performance at both line and paragraph levels, specifically analyzing how architectures scale with sequence length.
Efficiency vs. Accuracy Trade-off: Demonstrates that while accuracy differences are marginal at the line level, SSMs offer significant advantages in memory scaling and inference speed for long sequences.
Reproducibility: Releases code, trained models, and standardized gold-standard splits (corrected to >99% accuracy) for the research community.

4. Key Results

4.1 Line-Level Recognition

Accuracy: All neural models achieved low error rates (~1.8–2.6% CER). Mamba-AR and DAN tied for the best accuracy (1.83% CER on Antiqua).
Efficiency:
- Mamba-AR was 2.9× faster than DAN (53.3 ms vs. 156.5 ms per line) while maintaining identical accuracy.
- VAN (CTC-based) was the fastest overall (6.19 ms) but slightly less accurate than AR models on Fraktur.
- Off-the-shelf systems (Tesseract, zero-shot TrOCR) performed significantly worse on degraded historical text.

4.2 Paragraph-Level Recognition

Accuracy: DAN achieved the best accuracy (5.24% CER), followed closely by Mamba-AR (6.07% CER).
Speed: Mamba-AR was 2.05× faster than DAN (195.6 ms vs. 401.2 ms).
Memory Scaling (Critical Finding):
- As sequence length increased from 100 to 1000 characters, DAN's memory usage grew quadratically (2.30×) due to key-value cache expansion.
- Mamba-AR's memory usage grew linearly (1.26×), confirming the theoretical $O(n)$ complexity advantage.
- This allows Mamba to handle longer paragraphs with larger batch sizes without running out of GPU memory.

4.3 Tokenization Insights

Character-level tokenization proved sufficient for historical text.
BPE (DANIEL) showed a significant performance drop on Fraktur script (6.18% CER vs. 2.37% on Antiqua), suggesting that subword tokenizers trained on modern corpora struggle with archaic spellings and specific glyphs.

5. Significance and Conclusion

This paper establishes State-Space Models (Mamba) as a highly effective alternative to Transformers for historical document OCR, particularly for large-scale digitization projects.

Practical Impact: For institutions processing millions of pages, Mamba offers a "sweet spot" where accuracy remains competitive with the best Transformers (DAN), but inference is significantly faster and memory usage scales linearly.
Deployment: The linear memory scaling enables the processing of long paragraphs and full pages without the prohibitive memory costs associated with self-attention mechanisms.
Future Direction: The work shifts the focus from purely maximizing accuracy (which has plateaued at ~2% CER for neural models) to optimizing computational efficiency and scalability, guiding practitioners toward architectures that are feasible for mass deployment in cultural heritage digitization.

The authors conclude that while Transformers remain strong, Mamba-AR is the superior choice for balancing high accuracy with the operational constraints of speed and memory in long-sequence historical document recognition.