Towards Hierarchical Structure Understanding of Newspaper Images
This paper addresses the challenge of understanding complex newspaper layouts by proposing two complementary approaches—a modular bottom-up pipeline and a novel end-to-end transformer architecture called Tiramisu—while introducing a new dataset, Finlam La Liberté, to evaluate hierarchical information retrieval in historical newspapers.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine trying to read a giant, chaotic billboard that has been crumpled, stained with coffee, and then flattened out again. Now, imagine that billboard is actually a page from an old newspaper, packed with tiny headlines, long stories, weird ads, and pictures, all jumbled together without any clear lines telling you where one story ends and the next begins. This is the daily reality for computers trying to "read" history. For decades, scientists have been teaching machines to recognize words (a process called OCR) and to understand how a page is laid out. But old newspapers are a special kind of nightmare: they are dense, messy, and organized in a complex hierarchy where a big section contains smaller articles, which are made of paragraphs, which are made of words. If a computer gets the order wrong, it might read the end of a story before the beginning, or mix two different news stories into one giant, nonsensical paragraph. Making sense of this isn't just about typing out words; it's about understanding the structure of the page, like a librarian who knows exactly which book belongs on which shelf, even when the books are stacked in a pile.
This paper tackles that exact mess by testing two very different ways to teach a computer how to untangle a newspaper page. The researchers, working with the French National Library, wanted to see if they could build a system that could automatically figure out the layout, the reading order, and the content of these historical documents. They didn't just guess; they built two distinct "robots" to do the job and pitted them against each other.
The first robot is a Bottom-Up Pipeline. Think of this like a team of specialized detectives working in a line. First, one detective (a model called YOLO) scans the page and points out every single object: "That's a picture," "That's a title," "That's a paragraph." Next, a second detective (LayoutReader) looks at all those scattered objects and figures out the order in which you should read them, like connecting the dots. Finally, a third detective uses a custom rulebook to group those dots into complete stories and sections. It's a modular approach where each step relies on the previous one to pass the baton.
The second robot is a Top-Down Transformer named Tiramisu. Instead of a team of specialists, Tiramisu is a single, super-smart brain that looks at the whole page at once and tries to understand the hierarchy from the top down. It's like a master chef who doesn't just chop vegetables one by one but sees the entire meal and understands how the appetizer, main course, and dessert fit together before even touching the knife. Tiramisu works in "passes": first, it identifies the big sections; then, it zooms in to find the articles inside those sections; then, it finds the blocks inside the articles; and finally, it reads the text. It does all this in one go, learning the structure as it goes.
To test these two approaches, the team created a brand-new dataset called Finlam La Liberté, which contains 1,500 complete issues of a French newspaper from the 1920s. They also built a "synthetic" newspaper generator to create fake, perfect newspaper pages to help train the AI, since real historical pages are often too messy or damaged to teach a computer everything it needs to know.
The results were a tale of two very different strengths. The Bottom-Up Pipeline turned out to be the speedster and the precision expert. It was incredibly fast (taking less than a second on a powerful computer) and was the best at finding and labeling the tiny pieces of the puzzle, like individual paragraphs and images. It got the reading order of the tiny blocks right 87.20% of the time. However, it's a bit of a Rube Goldberg machine; if the first detective misses a spot, the whole chain can get confused.
Tiramisu, the all-in-one brain, was slower (taking about 1.5 seconds) and sometimes missed detecting small blocks entirely. If it missed a section in the first pass, it missed everything inside that section. However, when it did find things, it was excellent at understanding the big picture. It was surprisingly good at counting how many articles and sections were on a page, getting the "count" right 89% of the time, which is better than the pipeline. It also did a fantastic job at ordering the main stories, getting 97.43% of the article sequence correct.
The paper concludes that there is no single "perfect" solution yet. If you need to process millions of pages quickly and need to know exactly where every tiny paragraph is, the modular Bottom-Up Pipeline is the winner. But if you need a system that can understand the overall structure of a document in a single, unified model and are willing to trade a bit of speed and detection precision for that elegance, Tiramisu is a promising new direction. The researchers are so confident in the pipeline's speed and accuracy that they plan to use it to digitize over 8 million documents for the French National Library by the end of 2026, proving that sometimes, a team of specialists is the best way to solve a messy, historical puzzle.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.