← Latest papers
💬 NLP

MioFFAn: an Annotation Software for Formula Formalization with LLM Automation Capabilities

This paper introduces MioFFAn, an open-source, customizable annotation framework that extends the MioGatto architecture to facilitate the creation of high-quality datasets for formula formalization by combining human annotation with modular Large Language Model automation.

Original authors: Nicolas Sibuet, Horacio Saggion, Riccardo Rossi

Published 2026-07-28
📖 7 min read🧠 Deep dive

Original authors: Nicolas Sibuet, Horacio Saggion, Riccardo Rossi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but the clues are written in a secret code that only a computer can read, while the story is written in a language only humans understand. This is the daily struggle of scientists who write papers filled with complex math. They use beautiful, elegant formulas to describe how the world works, but these formulas are often just "pictures" of math—like a drawing of a bridge. To actually build the bridge, a computer needs the blueprints: the specific instructions, variables, and logic that tell a machine exactly how to calculate the stress on the steel. This process of turning a "picture" of math into "blueprints" for a computer is called Formula Formalization.

The problem is that computers, even the super-smart ones powered by Artificial Intelligence (AI), are terrible at reading these blueprints on their own. They need a massive library of examples—thousands of pairs of "math picture" and "computer blueprint"—to learn how to translate. But here's the catch: nobody has built this library yet because it is incredibly hard and boring to create. Humans have to sit down and manually rewrite every single math symbol from a paper into code, a task so tedious that it slows down scientific progress. It's like trying to teach a robot to cook by hand-writing every single recipe from scratch, one ingredient at a time, for every dish in the world.

This is where a new tool called MioFFAn steps in. Think of it as a high-tech, interactive notebook designed to make the boring job of translating math into code much faster and less painful. The researchers behind it didn't just build a static tool; they built a "human-in-the-loop" system. This means the software acts like a smart assistant that guesses the translation using AI, but then asks a human expert to double-check and fix the mistakes. It's like having a robot that can draft a letter for you, but you still have to sign it and make sure the facts are right before you send it. By using this team-up approach, the authors show that we can start building that crucial library of math-to-code examples much faster than before, opening the door for computers to finally understand and run the complex math that drives modern science.

The Paper's Story: Building a Bridge Between Math and Code

The paper introduces MioFFAn, a piece of open-source software designed to help researchers annotate scientific documents. Its main goal is to facilitate Formula Formalization, which is the act of taking a mathematical expression from a paper and turning it into executable symbolic code (like code you would write in a computer algebra system). The authors argue that while we have great tools to scan documents and read the text (OCR), we lack the "semantic depth" to turn those scanned formulas into code that a computer can actually run.

To solve this, the authors built MioFFAn on top of an older tool called MioGatto. They realized the old tool was great at identifying single math symbols (like the letter xx) but terrible at handling complex groups of symbols (like a whole function f(x)f(x) or a tensor δui\delta u_i) and didn't allow for the kind of flexible, hierarchical annotation needed for real-world science. MioFFAn fixes this by introducing several key upgrades:

  1. Grouping and Hierarchy: Instead of just tagging single letters, MioFFAn lets users select entire chunks of a formula as a single unit. It treats a complex expression like a "group" that can still be broken down into its parts if needed.
  2. Customizable Rules: The software is a chameleon. It allows users to define their own "taxonomies" (rules and categories) for different scientific fields. For example, a physicist might need to tag a variable as a "vector," while a biologist might need to tag it as a "concentration." The tool adapts to these needs.
  3. Contextual Grounding: It helps link the math symbols back to the text in the document where they are explained. If a paper says "let vv be the velocity," the tool helps you highlight that sentence and connect it to the symbol vv in the formula.
  4. The "Human-in-the-Loop" Automation: This is the most exciting part. The software includes a feature where an AI (specifically a Large Language Model, or LLM) tries to do the annotation first. It suggests which parts of the formula are which, what the variables mean, and where they are defined in the text. However, the human user isn't replaced; they act as a supervisor. They review the AI's suggestions, fix any errors, and confirm the final result.

The authors tested this system using a specific scientific field called Finite Element Methods (FEM), which is used for simulating things like how a car crashes or how a bridge bends. They set up the software to understand the specific math rules of FEM and then ran a "proof-of-concept" experiment. They fed the software some scientific papers and let the AI try to annotate them, then compared the AI's work against a "gold standard" created by human experts.

The results were a mix of promise and reality. The AI was surprisingly good at some things but struggled with others. For instance, when the AI was allowed to see the original mathematical symbols (like the Greek letter λ\lambda), it did a much better job of identifying the pieces of the formula (about 57.7% coverage) compared to when the symbols were replaced with long text names (which dropped to 36.2%). However, the AI was still not perfect; it often struggled to pinpoint the exact location of the text definitions in the document, sometimes highlighting whole paragraphs instead of just the specific sentence.

The paper explicitly rules out the idea that AI can currently do this job perfectly on its own. The authors state that the automated results are "far from being ideal" and should only be used as a "baseline" that a human expert rapidly refines. They argue that trying to use AI for a fully automatic, end-to-end translation without human checks would likely lead to errors that are hard to catch.

In their evaluation, the authors found that the "Original Characters" approach (keeping the math symbols as they are) worked best for the AI to understand the structure of the math. They also compared three different AI models (NVIDIA's Nemotron, Google's Gemma, and Alibaba's Qwen) and found that the Qwen3-4B-Instruct-2507 model performed significantly better than the others, achieving a 57.7% coverage rate in symbol identification compared to roughly 33.9% for the next best model.

Ultimately, the paper suggests that MioFFAn is a powerful "playground" for developing better automation strategies. It doesn't claim to have solved the problem of translating math to code forever. Instead, it offers a flexible framework where researchers can test different AI strategies, see what works, and gradually improve the system. The authors conclude that while the current automation is just a "provisional" step, this human-and-AI team-up is the most effective way to build the high-quality datasets needed to train future, smarter systems. They envision a future where this tool helps build a massive, shared library of math-to-code translations, making it easier for computers to help scientists solve the world's most complex problems.

Drowning in papers in your field?

Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.

Try Digest →