A FAIR-Compliant Data Lakehouse for Seismological Workflows: Architecture and Experimental Evaluation

The Earth is never truly still. Even in regions far from the dramatic collision zones where tectonic plates crash together, the ground constantly trembles with tiny, imperceptible shifts. In Brazil, scientists have spent decades listening to these whispers, deploying networks of sensitive instruments to record the subtle vibrations of the planet. This work generates a massive, continuous stream of data, a digital record of every tremor, big or small. However, managing this information has long been a struggle. The data arrives in many different formats, often stored in isolated pockets, and the process of turning raw signals into useful scientific knowledge has relied heavily on human hands. Researchers have had to manually move files, check spreadsheets, and piece together information from different sources, a method that is slow, prone to error, and difficult to repeat exactly.

To solve this, a team of researchers at the Federal University of Rio Grande do Norte in Brazil has built a new kind of digital home for this seismic data. They call it a "Data Lakehouse," a term that describes a system capable of storing vast amounts of raw information while also organizing it so it can be easily found, understood, and used by anyone. The goal was to create a single, unified framework that could handle the entire journey of a piece of data, from the moment it is captured by a sensor to the moment it becomes a scientific report. By automating the flow of information and keeping a strict, digital record of every step taken, the team aimed to make seismological research faster, more reliable, and transparent.

The researchers began by looking at how their own laboratory, LabSIS, currently operates. In the existing system, seismic data flows in from monitoring stations as raw digital files. A computer program processes these files to identify potential earthquake events, but the human element remains critical. Technicians must visually inspect the data, validate the events, and then manually transfer details into spreadsheets or share them through messaging apps. This fragmented approach means that the history of how a specific piece of information was created is often lost or scattered across different tools. To fix this, the team designed a new architecture that replaces these disjointed steps with a seamless, automated pipeline.

At the heart of this new system is a set of open-source tools working together like a well-coordinated crew. When data arrives, an automated system immediately grabs it and stores it in a massive, scalable digital warehouse. Instead of a human opening files one by one, the system uses specialized software to read the seismic signals, clean them up, and convert them into a standard format that everyone can understand. As this happens, the system automatically writes down a detailed log of every action it takes. It records where the data came from, what tools were used to process it, and what the final result looks like. This creates a complete, unbroken chain of evidence, known as data lineage, which allows anyone to trace a final scientific finding back to the original sensor reading.

To test if this idea worked in the real world, the team ran a large-scale experiment using actual data from the Brazilian Seismographic Network. They fed the system 825 raw data files, representing about 6.6 gigabytes of information collected over three weeks. The entire process, from ingestion to the creation of organized, ready-to-use datasets, took just under 58 minutes. This was a significant improvement over the traditional method, which would have required hours of manual coordination and was far more likely to introduce mistakes. The system successfully processed the data without any human intervention, demonstrating that it could handle the continuous, high-volume stream of information that modern seismology demands.

The researchers also invited a group of ten experts from the fields of seismology, computer science, and geophysics to evaluate the new system. These specialists, ranging from early-career scientists to senior researchers with decades of experience, tested the architecture against a set of criteria focused on how easy it was to find data, how well it was organized, and whether it followed the principles of "FAIR" science. FAIR stands for Findable, Accessible, Interoperable, and Reusable, a global standard for making scientific data useful to everyone. The experts rated the system very highly, giving it an average score of 4.6 out of 5. They particularly praised the system's ability to keep track of data origins and its automated nature, noting that it would significantly reduce the time they spent on administrative tasks and allow them to focus more on scientific analysis.

One of the most compelling aspects of the new system is its ability to tell the full story of a single earthquake event. In a specific test case involving a magnitude 4.5 earthquake in Itapé, Brazil, the researchers showed how the system linked the raw sound of the ground shaking to the final map and report. The system recorded the exact moment the data was captured, the specific station that recorded it, the software used to analyze it, and the final product generated. This level of detail ensures that if another scientist wants to verify the results, they can do so with complete confidence, knowing exactly how the data was handled. This transparency is a crucial step toward making scientific research more reproducible and trustworthy.

While the results are promising, the researchers are careful to note that this is a proof of concept rather than a finished product for every possible scenario. The system was tested on a single local computer rather than a massive global network, and it currently handles data after it has been collected rather than in real-time as it happens. The team also acknowledges that while the system is excellent at organizing data, there is still work to be done to make the meaning of that data perfectly clear to every computer system in the world. Despite these limitations, the study demonstrates that it is possible to build a system that brings order to the chaos of modern seismic data. By combining powerful automation with strict rules for how data is managed, the researchers have created a foundation that could transform how scientists study the Earth, turning a slow, manual process into a fast, reliable, and transparent operation.