LaTeX Compilation: Challenges in the Era of LLMs

This paper critiques the limitations of TeX in the era of large language models and introduces Mogan STEM, a WYSIWYG structured editor that offers superior compilation efficiency, error localization, and lower information entropy for more effective LLM fine-tuning.

The Gist

Here is an explanation of the paper, translated into simple language with everyday analogies.

The Big Picture: An Old Map vs. A GPS

Imagine you are trying to write a scientific paper. For decades, the standard tool for this has been LaTeX (based on the older TeX system). Think of LaTeX like an old, manual map. It's incredibly precise and has been used by explorers for 40 years, but it has some major flaws:

1. It's slow to update: If you change one street name on the map, you have to redraw the entire map from scratch to see if the route still works.
2. It's confusing: If you make a small mistake (like a missing comma), the map doesn't tell you where you went wrong; it just says, "The whole trip failed," and you have to guess which part broke it.
3. It's heavy: To use this map, you need to carry a massive backpack full of every possible road sign, font, and rulebook, even if you only need to walk to the corner store.

The authors of this paper argue that in the age of AI (Large Language Models or LLMs), this old map is becoming a burden. They propose a new tool called Mogan STEM, which is like a modern GPS. It updates instantly, tells you exactly where you are, and only loads the features you need right now.

---

The Problems with the Old Way (LaTeX)

The paper breaks down three main headaches caused by LaTeX:

1. The "Rewrite Everything" Problem (Batch Processing)

• The Analogy: Imagine you are writing a story. In the old way, every time you delete a paragraph, the computer has to re-read your entire story from page 1 to the end, just to see if your page numbers are still correct.
• The Reality: LaTeX works in "batches." It reads your code, does the math, and spits out a PDF. If you change one word, it doesn't just update that word; it re-runs the whole process. This makes it slow and frustrating when you want instant feedback.

2. The "Blindfolded" Problem (Error Localization)

• The Analogy: Imagine you are building a Lego castle. You forget to put a brick in the foundation. In the old way, the castle doesn't collapse immediately. You keep building the tower, and only when you try to put the flag on top does the whole thing fall over. The computer then says, "Error! The flag won't stick!" but it doesn't tell you that the foundation was missing.
• The Reality: LaTeX often hides errors. A small typo in the first paragraph might cause a crash in the last paragraph. The error message points to the crash site, not the cause, making it a nightmare to fix.

3. The "Backpack" Problem (Installation & Complexity)

• The Analogy: To use LaTeX, you have to download a "kitchen" that includes every possible appliance (toaster, blender, industrial oven) even if you just want to make toast.
• The Reality: The LaTeX installation is huge (growing to nearly 6GB). It requires complex setup and multiple steps just to get a simple document working.

---

The New Solution: Mogan STEM

The authors introduce Mogan STEM, a new editor designed to fix these issues.

1. The "Living Tree" Structure

• The Analogy: Instead of a long scroll of text (LaTeX), Mogan organizes your document like a family tree or a flowchart. Every section, formula, and image is a distinct "node" (a branch).
• The Benefit: If you change a branch (like a math formula), the computer only recalculates that specific branch. It doesn't touch the rest of the tree. This makes it instantly fast.

2. "What You See Is What You Get" (WYSIWYG)

• The Analogy: With LaTeX, you write code like $$\frac{1}{2}$$ and hope it looks right. With Mogan, you just type the fraction, and it looks like a fraction immediately, just like in Microsoft Word, but with the superpowers of a math editor.
• The Benefit: You never have to guess if your code is right. If you make a mistake, the editor catches it right there, so you don't have to wait for a "crash" to find out.

3. The "On-Demand" Backpack

• The Analogy: Instead of carrying a backpack full of tools, Mogan is like a smart toolbox. It only pulls out the hammer when you need to hammer a nail, and the screwdriver when you need to screw something.
• The Benefit: The software is tiny (about 100MB) and starts up instantly. It doesn't waste space on things you aren't using.

---

Why This Matters for AI (The "Secret Sauce")

This is the most exciting part of the paper. The authors tested how well AI models (like the ones writing this summary) understand these two formats.

• The Finding: AI learns much better from Mogan files than LaTeX files.
• The Analogy:
  • LaTeX is like a messy recipe written in a secret code: "Add 2 cups of flour, then if the oven is hot, add 1 egg, but if the flour is from Brand X, add 2 eggs." It's full of noise and confusing instructions.
  • Mogan is like a clean, structured recipe card: "Step 1: Flour. Step 2: Egg."
• The Result: Because Mogan is structured and logical (like a tree), the AI can predict what comes next much more easily. The paper shows that when training AI to write math or science, using Mogan files makes the AI smarter, faster, and less likely to make mistakes.

The Bottom Line

The paper argues that while LaTeX was a hero in the 1980s, it is holding us back today. It's too slow, too confusing, and too heavy for the modern era of instant feedback and AI collaboration.

Mogan STEM is the upgrade we need: it's fast, it's easy to use, it fixes errors instantly, and it speaks the language that AI understands best. The authors are calling for scientists and developers to start switching to this new format to make writing and AI training more efficient.

Technical Summary

Here is a detailed technical summary of the paper "LATEX Compilation: Challenges in the Era of LLMs" by Liu et al.

1. Problem Statement

The paper argues that while TeX/LaTeX has been the de facto standard for scientific typesetting for decades, its underlying architecture is fundamentally ill-suited for the modern era of Large Language Models (LLMs) and interactive, real-time writing. The authors identify three core categories of limitations:

• Compilation Architecture Defects:
  • Batch Processing & Unidirectionality: TeX uses a linear, one-pass (or multi-pass) batch model. It lacks an intermediate structured state, meaning local edits require full re-compilation (no incremental updates).
  • Delayed Feedback: Errors (e.g., unclosed braces) often manifest far from their source, sometimes only after multiple compilation passes, making debugging difficult.
  • Weak Semantics: Syntax parsing, macro expansion, and typesetting are tightly coupled. Commands like \section conflate structural semantics with layout instructions, preventing static analysis tools from understanding document structure without executing the code.
• User Experience & Ecosystem Issues:
  • Fragmentation: Multiple engines (pdfLaTeX, XeLaTeX, LuaLaTeX) exist with incompatible features (e.g., Unicode support), forcing users to manage complex workflows.
  • Bloat: Distributions like TeX Live have grown to ~6GB, with massive overhead for basic users.
  • Tool Limitations: Real-time previews in editors are merely simulations of background compilations, leading to latency and layout inconsistencies.
• LLM Inefficiency:
  • High Entropy: LaTeX syntax is noisy and ambiguous (e.g., multiple ways to write the same math), increasing the token cost and confusion for LLMs.
  • Structural Opacity: LLMs struggle to parse document structure (references, sections) because the logical structure is hidden behind macro expansions and requires multiple compilation passes to resolve.

2. Methodology & Proposed Solution: Mogan STEM

The authors introduce Mogan STEM, a WYSIWYG (What You See Is What You Get) structured editor based on GNU TeXmacs, as a superior alternative.

• Core Architecture:
  • Tree-Based Representation: Unlike LaTeX's linear token stream, Mogan represents documents and formulas as structured trees (using Scheme S-expressions). For example, a fraction is stored as (frac "1" "2") rather than \frac{1}{2}.
  • Functional Rendering: Mathematical symbols are treated as functions (e.g., scalable brackets) rather than static strings.
  • On-Demand Loading: Instead of a massive pre-installed package repository, Mogan uses a monolithic core with plugins loaded dynamically only when specific document structures trigger them.
• Experimental Design:
  • Performance Benchmarks: Compared compilation/rendering times and incremental update speeds between LaTeX and Mogan STEM across 6 real-world arXiv papers.
  • LLM Task Evaluation: Tested 4 state-of-the-art LLMs (Deepseek, Claude, Gemini, ChatGPT) on three tasks:
    1. Locating document structure.
    2. Merging files with conflicting document styles (macros/packages).
    3. Debugging ill-formed documents using error messages.
  • Fine-Tuning Efficiency: Conducted a Supervised Fine-Tuning (SFT) experiment using LoRA on Qwen2.5-7B to predict the next token in LaTeX vs. Mogan formats.

3. Key Contributions

1. Critical Analysis of TeX: A systematic deconstruction of TeX's architectural flaws (batch processing, lack of incremental updates, semantic coupling) that hinder modern AI integration.
2. Mogan STEM Introduction: A demonstration of a structured, tree-based editor that decouples content from layout, enabling instant rendering and error isolation.
3. Empirical Evidence for LLMs:
   • Lower Information Entropy: Mogan's .tmu format (structured tree) is more deterministic and less ambiguous than LaTeX, making it easier for models to learn and predict.
   • Error Localization: Errors in Mogan are localized to specific tree nodes, whereas LaTeX errors often propagate globally, confusing LLMs.
4. Fine-Tuning Results: Demonstrated that fine-tuning LLMs on Mogan data results in faster convergence and lower loss compared to LaTeX data due to reduced syntactic noise.

4. Experimental Results

• Compilation/Rendering Speed:
  • Mogan STEM outperformed LaTeX in incremental updates across all test documents (up to 10x faster in some cases) because it only re-renders affected tree nodes.
  • For full compilation, Mogan was competitive or faster, though large documents (120+ pages) showed higher I/O overhead due to the WYSIWYG nature.
• LLM Performance:
  • Structure Locating: Mogan files allowed LLMs to locate sections and references significantly faster (higher utility scores) because references are stored as explicit structural links rather than requiring compilation to resolve.
  • File Merging: Mogan achieved near-perfect scores in merging conflicting document styles. LLMs struggled with LaTeX due to macro aliasing and package conflicts, whereas Mogan's consistent grammar eliminated these ambiguities.
  • Debugging: LLMs solved all Mogan error cases quickly. In contrast, LLMs spent excessive tokens trying to debug LaTeX errors because the error logs were long and detached from the root cause.
• Fine-Tuning (LoRA):
  • On a dataset of 1,000 mathematical formulas, the model trained on Mogan data converged to a loss of ~0.4, while the LaTeX-trained model converged to ~0.7.
  • The authors attribute this to Mogan's lower information entropy and lack of "syntax noise" (e.g., LaTeX allows multiple equivalent syntaxes for the same output, confusing the model).

5. Significance

The paper posits a paradigm shift for scientific writing in the AI era:

• From Batch to Interactive: The move away from batch compilation toward structured, incremental editing is essential for real-time collaboration and AI-assisted writing.
• AI-Native Formats: The .tmu format (or similar structured tree formats) is presented as a superior training data format for LLMs. It reduces token costs, improves reasoning accuracy regarding document structure, and accelerates model convergence.
• Future of Tooling: The authors call for a transition away from the "legacy debt" of TeX. They argue that while Overleaf and other platforms optimize the interface of LaTeX, they do not solve the architectural defects. A new generation of structured editors (like Mogan) is required to fully leverage the capabilities of modern LLMs.

In conclusion, the paper suggests that TeX is a historical artifact that, while powerful for static typesetting, is a bottleneck for dynamic, AI-driven scientific workflows. Adopting structured, tree-based formats like Mogan STEM offers immediate benefits in speed, usability, and AI compatibility.