HPD-Parsing: Hierarchical Parallel Document Parsing
HPD-Parsing introduces a hierarchical parallel decoding paradigm that combines global layout analysis with concurrent block-level content generation and progressive multi-token prediction, achieving a 4,752 tokens-per-second throughput (2.62x faster than existing models) while maintaining competitive accuracy.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are trying to read a massive, complex library book that has been glued shut. To understand it, you have to read every single word from the very first page to the very last, one by one, without ever skipping ahead. This is how many modern computer programs currently try to "read" documents like PDFs or scanned papers. They use a type of artificial intelligence called a Vision-Language Model (VLM), which acts like a super-smart robot that can see images and read text at the same time. While these robots are getting incredibly good at understanding what a document says, they are often painfully slow. It's like trying to solve a giant jigsaw puzzle by only looking at one piece at a time, waiting for the previous piece to be placed before you can even touch the next one. As documents get longer and more crowded with text, tables, and math formulas, this "one-by-one" method becomes a traffic jam, making it hard to process thousands of documents quickly.
This is where a new idea called HPD-Parsing comes in. The researchers behind this project realized that while a document needs a global plan (like knowing the order of chapters), the actual reading of each section doesn't need to happen in a strict line. They propose a smarter way to work: instead of one robot reading the whole book sequentially, they use a team. One "manager" robot figures out the layout and points to different sections, while a team of "worker" robots reads those sections all at the same time. They also added a trick where the robots can guess several words ahead at once, skipping the need to stop and think after every single word. The result is a system that is dramatically faster—processing over 4,752 words every second—without losing any of its ability to understand the document correctly.
The Problem: The Slow, Single-File Line
Think of a traditional document parser like a single person trying to eat a giant, multi-course banquet. They have to finish the soup before they can touch the salad, and the salad before the main course. Even if the soup is simple, they can't start the salad until the soup is gone. In the world of computer science, this is called autoregressive generation. The computer generates the output (the text it reads out of the document) one token (a tiny piece of a word) at a time. It looks at what it just wrote, decides what comes next, writes that, and repeats.
For short notes, this is fine. But for a 50-page document full of charts, math equations, and dense text, this single-file line creates a massive bottleneck. The computer spends most of its time waiting for itself to finish the previous step before it can take the next one. The researchers found that for long documents, the time spent decoding the text was nearly 500 times longer than the time spent just looking at the picture of the page. It's like spending five hours driving to the grocery store just to spend one minute picking out an apple.
The Solution: A Team of Super-Readers
The authors of this paper, HPD-Parsing, decided to break the single-file line. They introduced a concept called Hierarchical Parallel Decoding. Imagine a construction site where a foreman (the "Layout Branch") stands on a scaffold looking at the whole building. The foreman doesn't lay every brick; instead, they point to different sections of the wall and say, "You, build the kitchen! You, build the bedroom! You, build the bathroom!"
In this new system:
- The Manager (Layout Branch): This part of the AI looks at the whole document image first. It figures out the structure: "Here is a title, here is a paragraph, here is a table, and here is a math formula." It creates a map of the document.
- The Workers (Content Branches): As soon as the manager identifies a section, it spawns a new, independent "worker" AI to read just that section. Crucially, these workers all start reading their assigned sections at the same time. They don't wait for the kitchen to be finished before the bedroom starts.
- Shared Memory: To save time, all these workers share the same "memory" of the original image and the manager's map. They don't need to re-read the whole picture; they just focus on their specific job.
The Secret Weapon: Guessing the Future
Even with a team of workers, reading one word at a time is still a bit slow. So, the researchers added a second layer of speed called Progressive Multi-Token Prediction (P-MTP).
Imagine you are reading a sentence: "The cat sat on the..."
A normal reader stops after "on" and thinks hard about what comes next. They might guess "mat." Then they stop again to think about the next word.
The P-MTP system is like a reader who looks at "The cat sat on the" and confidently guesses the next three words at once: "mat, and, slept." It then checks if those guesses are right. If they are, it writes them all down in one go. If not, it corrects itself and tries again.
In the HPD-Parsing system, every single worker (and the manager) uses this trick. Instead of taking one step at a time, they take big leaps, predicting several words ahead. The paper reports that, on average, this allows the system to accept about 6.6 words in a single step, rather than just one.
The Results: Fast and Accurate
The researchers tested this new system on a standard benchmark called OmniDocBench V1.6, which includes all sorts of tricky documents with complex layouts, math, and tables.
- Speed: The new HPD-Parsing system achieved a speed of 4,752 tokens per second. This is 3.06 times faster than the standard "one-by-one" method and 2.62 times faster than the fastest existing document parser currently available.
- Accuracy: Despite being so much faster, the system didn't get sloppy. It maintained a competitive accuracy score of 94.91, which is actually higher than many other powerful models that are much larger and slower.
- Handling Errors: The team showed that this method is also more robust. If a traditional system makes a mistake early on, it often gets confused and repeats the same error over and over for the rest of the document. Because HPD-Parsing splits the work into independent branches, a mistake in one section (like a table) stays in that section and doesn't ruin the rest of the document.
Why It Matters
This paper suggests that we don't have to choose between speed and accuracy. By realizing that documents have a natural structure—a global layout that can be managed by one brain, while local content can be read by many brains working together—we can process information much more efficiently.
The researchers didn't just build a faster computer; they changed the way the computer thinks about reading. Instead of a lonely, slow walker, they built a coordinated team of sprinters. This approach, they argue, opens the door to processing massive libraries of documents in real-time, making it possible for AI to help us with information extraction, research, and data retrieval at a scale that was previously impossible. The paper concludes that this "hierarchical parallel" style is a powerful new direction for the future of document parsing.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.