Texo: Formula Recognition within 20M Parameters

This paper introduces Texo, a highly efficient 20-million-parameter formula recognition model that achieves state-of-the-art performance with significantly reduced size, enabling real-time inference on consumer hardware and in-browser deployment.

The Gist

Imagine you have a giant, super-smart library of mathematical formulas. For a long time, the only way to read these formulas (turning a picture of an equation into computer code) was to hire a massive, expensive team of experts. These "experts" were huge computer models that required powerful, industrial-grade supercomputers to run. They were like a 10-ton truck trying to deliver a single letter; they could do the job, but they were slow, expensive, and impossible to fit in a regular garage (your home computer).

This paper introduces Texo, a new model that changes the game. Think of Texo not as a 10-ton truck, but as a sleek, high-speed electric scooter. It's tiny, incredibly efficient, and can zip through the same job just as well as the giant truck, but it fits right in your pocket.

Here is the story of how they built this "scooter" using three clever tricks:

1. The Problem: The "Over-Engineered" Dictionary

The old models (like the giant trucks) were built with a dictionary meant for the entire English language. They knew words like "elephant," "butterfly," and "democracy." But when you are reading math, you don't need to know about butterflies! You only need to know about symbols like +, ∫, √, and π.

Because these old models carried around a dictionary of 50,000 words, they were bloated and heavy. It was like trying to carry a whole encyclopedia in your backpack just to read a single math textbook.

The Texo Solution:
The authors realized that math has its own strict language. They created a specialized, mini-dictionary containing only the 687 most important math symbols.

• The Analogy: Instead of carrying a library, Texo carries a pocket cheat sheet. By throwing out the 49,313 useless words, they shaved off 80% of the model's weight instantly.

2. The Transfer: "Stealing" the Brain, Not the Body

Usually, when you want to build a smart AI, you have to train it from scratch, which takes forever and needs a lot of data. Texo didn't start from zero. The team took a slightly smaller, pre-trained model (called PPFormulaNet-S) and performed a "brain transplant."

They kept the smart parts that knew how to look at an image and understand shapes, but they swapped out the heavy, clunky vocabulary for their new, tiny math dictionary.

• The Analogy: Imagine taking a professional race car engine (the smart brain) and putting it into a lightweight, aerodynamic chassis (the new vocabulary). The engine knows how to drive fast, but now the whole car is light enough to fly.

3. The Result: Math in Your Browser

The final result is a model with only 20 million parameters (a tiny number in the AI world).

• Speed: It is 7 times faster than the previous best open-source model.
• Accessibility: Because it is so small, you don't need a supercomputer. You can run it right in your web browser.
• Privacy: The authors built a website where you can upload a photo of a formula, and the math is processed entirely on your own computer. No data is sent to a server. It's like doing your homework in your own living room rather than sending it to a school office where someone else might read it.

Why Does This Matter?

In the past, if you wanted to turn a picture of a complex equation into editable text, you had to pay for expensive software or use slow, heavy tools.

Texo proves that you don't need a "giant" to do a "giant" job. By being smart about what the model learns (focusing only on math) and how it speaks (using a tiny, efficient dictionary), they achieved the same high quality as the giants, but with a fraction of the resources.

In short: Texo is the "Swiss Army Knife" of formula recognition. It's small, fits in your pocket, does everything the big tools do, and you can use it for free, right now, without worrying about your privacy.

Technical Summary

1. Problem Statement

Mathematical Expression Recognition (MER) is a critical task for document analysis, converting formula images into structured LaTeX code. While recent advancements in Vision-Language Models (VLMs) and Transformer-based architectures (e.g., UniMERNet, PPFormulaNet) have achieved State-of-the-Art (SOTA) performance, they suffer from high parameter counts (ranging from 58M to over 500M).

• Limitations: These large models are computationally expensive, requiring high-end GPUs (e.g., A100s) for training and inference. This makes them unsuitable for consumer-grade hardware, edge devices, or real-time in-browser applications.
• Inefficiency: Existing models often inherit vocabularies and tokenizers designed for general natural language processing (NLP). These are suboptimal for LaTeX, which has rigorous syntax and specific semantics, leading to wasted parameters in embedding layers and inefficient tokenization (e.g., splitting meaningful LaTeX macros like \leftarrow into subwords).

2. Methodology

The authors propose Texo, a minimalist model designed to achieve SOTA performance with only 20 million parameters. The methodology focuses on architectural efficiency and specialized vocabulary engineering.

A. Model Architecture

Texo inherits a lightweight architecture from PPFormulaNet-S:

• Image Encoder: Uses HGNetV2-B4, a lightweight CNN backbone (originally for RT-DETR), which is efficient for image feature extraction.
• Text Decoder: A 2-layer MBart Transformer decoder with a hidden dimension of 384 and a context length of 1024.
• Mechanism: The decoder performs cross-attention on the encoded image features to output log-probabilities for sequence generation.

B. Vocabulary Distillation and Transfer (Key Innovation)

The primary driver of parameter reduction is the redesign of the vocabulary and tokenizer:

• Problem Identification: Standard BPE tokenizers used in general NLP models create large vocabularies (e.g., 50k tokens) and split LaTeX macros into meaningless subwords, increasing sequence length and embedding parameters.
• Solution:
  1. Rule-Based Tokenization: The authors built a custom tokenizer based on KaTeX (a LaTeX parser) to capture complete LaTeX macros as single tokens.
  2. Vocabulary Distillation: The vocabulary size was reduced from 50,000 to 687 tokens by removing whitespace and focusing strictly on LaTeX syntax.
  3. Knowledge Transfer: Using a heuristic transfer method (Algorithm 1), the input and output embedding layers of the pre-trained PPFormulaNet-S were mapped to the new, smaller vocabulary. The embeddings for new tokens were initialized by averaging the embeddings of their constituent tokens from the original model.
• Impact: This reduced the embedding parameters from 38M to <1M, bringing the total model size down to 20M.

C. In-Browser Deployment

To demonstrate efficiency, the model was deployed purely on the client-side (frontend):

• Converted to ONNX format and served via Transformers.js.
• Uses Web Workers to separate inference from UI rendering.
• Features include conversion to Typst/MathML and a WYSIWYG editor, ensuring privacy (no data leaves the user's device) and zero backend API costs.

3. Key Contributions

1. Texo Model: A 20M parameter formula recognition model that rivals SOTA models (UniMERNet-T and PPFormulaNet-S) in accuracy.
2. Vocabulary Engineering: A novel approach to distill and transfer vocabulary specifically for LaTeX, reducing model size by 80% compared to UniMERNet-T and 65% compared to PPFormulaNet-S.
3. Accessibility: Demonstration of high-performance MER on consumer-grade hardware (e.g., RTX 3090) and in web browsers without backend dependencies.
4. Open Source: Release of the model weights, training pipeline, and a functional web application.

4. Experimental Results

The model was evaluated on the UniMER-Test dataset (23k+ samples) using the Character Detection Matching (CDM) score, the current de-facto standard for MER.

Model	Parameters	CDM Score (Avg)	Inference Speed (vs. UniMERNet-T)
UniMERNet-T	107M	High (SOTA)	Baseline (1x)
PPFormulaNet-S	58M	Moderate	~3x faster (due to parallel decoding)
Texo	20M	Comparable/Higher	~7x faster than UniMERNet-T

• Accuracy: Texo outperformed its base model (PPFormulaNet-S) across all categories (Simple, Complex, Screen-Captured, Handwritten). For example, on Handwritten Expressions (HWE), Texo achieved a CDM of 0.902 vs. PPFormulaNet-S's 0.818.
• Efficiency:
  • Inference Speed: Texo is 7x faster than UniMERNet-T. While slightly slower than PPFormulaNet-S (which uses multi-token parallel prediction), Texo achieves this without sacrificing accuracy for speed.
  • Token Length: The curated tokenizer reduced output token length by nearly 50%, further accelerating inference.
• Hardware: Training was feasible on a single A40 46GB GPU (or even an RTX 3090 with gradient accumulation), whereas SOTA models often require 8x A100s.

5. Significance

• Democratization of MER: Texo proves that specialized, high-performance models do not require massive parameter counts. It challenges the notion that "bigger is better" for domain-specific tasks.
• Edge and Web Deployment: By reducing the model to 20M parameters, MER becomes viable for real-time applications on personal computers and directly in web browsers, enhancing privacy and reducing infrastructure costs.
• Methodological Insight: The paper highlights that vocabulary design is a critical, often overlooked factor in model efficiency. Tailoring tokenizers to the specific syntax of the domain (LaTeX) yields greater parameter savings than simply shrinking the neural network layers.
• Future Outlook: The authors suggest that this parameter-efficient approach can be extended to broader document OCR tasks, bridging the gap between general-purpose VLMs and specialized, lightweight models.