Newspapers are the daily record of human life, capturing everything from local elections to the price of flour, from obituaries to advertisements for new inventions. For centuries, these stories have been preserved on fragile paper, but in the digital age, libraries have scanned millions of these pages to make them accessible. However, turning a picture of a newspaper page into useful, searchable text is surprisingly difficult. The pages are dense and messy, filled with columns of text, decorative headlines, and advertisements that often bleed into one another. Standard computer programs, which work well on clean documents, often get confused by this chaos, producing text that is full of errors or missing entire sections. This creates a barrier for researchers and artificial intelligence systems that want to learn from history, as they cannot easily read what is written on the page.

A team of researchers from Harvard Law School and the Boston Public Library has developed a new way to solve this problem. They created a step-by-step system designed to break down a single newspaper page into its smallest, most logical pieces, read the text in each piece with high accuracy, and then organize everything so that a computer can understand the layout and content. Their goal was not just to digitize the text, but to create a clean, structured dataset that reveals the true content of over a million historical newspaper scans. By treating each distinct block of text or image as its own separate item, they managed to extract billions of words from materials that were previously too messy to use effectively.

The process begins by taking a digital image of a newspaper page and teaching a computer to see it the way a human reader does. Instead of trying to read the whole page at once, the system first identifies every distinct block of content, such as a single article, a photograph, or an advertisement. The researchers trained a computer model to recognize these blocks, or "crops," by showing it thousands of examples. This model learned to ignore the empty white space around the edges and the complex borders between columns, focusing only on the actual content. When tested, this segmentation tool proved highly effective, successfully identifying over 83 million individual blocks across more than 1.4 million newspaper pages. On average, each page was broken down into about 56 separate pieces, allowing the system to handle the complex layouts of the past with precision.

Once the page is divided into these manageable pieces, the system reads the text within each one. The researchers used two different methods to do this, running them side by side to ensure the best results. The first method uses a traditional tool that has been used for decades to convert images of text into digital letters. The second method uses a newer, more advanced type of artificial intelligence model that can understand the context of an image better. This newer model proved particularly good at reading text that was damaged, faded, or written in decorative fonts that confused the older tool. In many cases, the advanced model could even guess missing letters in a torn section of a page, filling in the gaps to create a complete sentence. By combining the outputs of both methods, the team generated a massive collection of text, totaling 16.3 billion tokens, which is enough to fill thousands of books.

With the text extracted, the system then works to understand what each piece of content actually is. It sorts the millions of blocks into categories, distinguishing between news articles, advertisements, photographs, and cartoons. To do this accurately, the system looks at both the visual appearance of the block and the words it contains. For example, a block that looks like a photograph but contains no text is identified as an image, while a block with a headline and a story is identified as news. The researchers found that advertisements made up the largest number of these blocks, but the actual news stories contained the majority of the words. This distinction is crucial because it allows researchers to study how the visual nature of newspapers changed over time, noting that advertisements became increasingly visual and took up more space on the page as the decades passed.

The system also organizes the text in the order a human would naturally read it. Newspapers often have articles that start in one column and continue in another, or stories that are interrupted by pictures. The researchers developed a method to map out these paths, determining which block comes next and which comes after that. While this is a complex task, their approach successfully reconstructed the reading order for a significant portion of the pages, achieving an accuracy of 72.1% on a macro level and 80.8% on a micro level, allowing the text to flow logically from start to finish. This step transforms a jumbled collection of image fragments into a coherent narrative that can be searched and analyzed just like a modern document.

Beyond simply reading and organizing the text, the system adds layers of useful information to every single block. It identifies the language used, finding that while most of the collection is in English, there are significant portions in Yiddish, German, Swedish, and French, reflecting the diverse immigrant communities of the time. It also scans the text to find names of people, places, and organizations, tagging millions of mentions of locations like Boston and New York, or institutions like Congress. Furthermore, the system assigns a topic to each block, labeling some as business reports, others as political news, and others as scientific updates. This rich tagging allows researchers to ask specific questions, such as "How often was the word 'Boston' mentioned in advertisements compared to news stories?" or "What topics were most common in the 1880s?"

The entire operation was designed to be efficient enough to run on standard computer hardware, rather than requiring massive, expensive supercomputers. The researchers processed the data in batches, optimizing each step to ensure that the system could handle the sheer volume of material without crashing or becoming too slow. The total cost to run this pipeline on rented computer power was estimated at around 25,000 dollars, a fraction of what it would have cost to use the most advanced, expensive models available. This efficiency means that other libraries and institutions could potentially use the same tools to unlock their own historical collections.

The result of this work is a massive, open dataset that includes not just the text, but the structure and context of over a million newspaper pages from between 1795 and 1930. The researchers have made the tools, the models, and the final data available to the public, allowing anyone to explore this history in new ways. While the system is not perfect and still requires human oversight for the most difficult cases, it represents a significant leap forward in making historical newspapers accessible. By turning chaotic, noisy images into clean, structured data, the team has opened a window into the daily lives of people from the past, providing a foundation for future research and a new way for artificial intelligence to learn from human history.