Open Science with the LHCb Ntupling Service

High-energy physics is the study of the universe's most fundamental building blocks, conducted by smashing particles together at speeds close to light. These collisions create a chaotic spray of new particles, which detectors capture in vast amounts of digital information. For decades, the scientists who built these massive machines were the only ones who could read the data, because the files were written in complex, custom languages that required specialized software to decode. This created a barrier: once a team finished its official analysis, the raw data often sat idle, inaccessible to the wider world. The goal of "open science" is to change this, allowing researchers, students, and curious minds anywhere to revisit these collisions and ask new questions. However, simply releasing the raw files is not enough; the data is too large and too complicated for anyone without a PhD in the specific experiment to use. The challenge has been finding a way to make this treasure trove of information usable without forcing every new user to learn the entire, intricate history of the experiment's software.

To solve this, the LHCb collaboration at CERN has built a new digital tool called the LHCb Ntupling Service. This is a web-based platform that acts as a bridge between the massive, complex data stored in the Large Hadron Collider and the everyday tools scientists use to analyze it. Instead of forcing users to download terabytes of raw, unreadable files, the service allows them to request exactly the information they need. A user can log in, select a specific type of particle collision they are interested in, and choose which details about those collisions they want to see. The system then automatically processes the data, extracting only those specific details and organizing them into a clean, standard format that can be opened with common analysis software. This process, known as "ntupling," transforms the data from a locked vault into a manageable, open book. The service covers data from the first two major runs of the collider, representing over 4 petabytes of collision history, and for the first time, it makes the data from the second run accessible in this flexible way.

The system works by connecting a user-friendly website to a powerful computing grid that runs across the globe. When a person visits the site, they are greeted with a gallery of examples, showing how others have used the tool to study things like the mass of specific particles. If they want to create their own study, they use a guided interface to pick the particles they want to investigate, such as a specific decay process, and then select the exact variables they wish to record, like the speed or direction of the particles. The system does not accept code or instructions that could harm the network; instead, it translates the user's choices into a safe, standard configuration file. This file is then sent to a team of experts who review it to ensure it is correct and safe. Once approved, the request is sent to the Worldwide LHC Computing Grid, where thousands of computers work together to process the data. The system first runs a small test to make sure the user gets the right information. If the test is successful, the user can approve a full run, which processes the entire dataset they requested.

The result is a custom data file, typically ranging from a few hundred megabytes to a few hundred gigabytes, which the user can download directly from the web. This file contains only the information the user asked for, stripped of the unnecessary complexity of the original experiment. This approach offers a significant advantage over previous methods. In the past, to make data open, teams had to release the entire dataset in its original, massive format, which for the second run of the collider would have meant copying nearly 10 petabytes of data to a public portal—a task that was too expensive and resource-heavy to be feasible. By creating these custom, smaller files on demand, the service keeps the data footprint small, with the first 100 requests generating only about 10 terabytes of total output. This makes the data accessible to anyone with a standard computer and internet connection, removing the need for expensive computing clusters or specialized training.

The service is designed to be intuitive, even for those with no background in particle physics. It includes detailed guides and examples that walk users through every step, from selecting a dataset to analyzing the results. Users can communicate directly with the LHCb open data team through the platform, asking questions or getting help with their specific requests. This interaction has proven valuable, with the team receiving positive feedback from a diverse group of users, including university students, high school teachers, and professional researchers. The tool has already been used for a wide variety of purposes, from studying the properties of exotic particles to developing new statistical methods and teaching physics concepts. The system is also linked to the CERN Open Data Portal, where successful requests can be turned into permanent, citable records with unique digital identifiers, ensuring that the data remains available for future generations.

Looking ahead, the service is designed to grow. The team plans to include data from 2018, which was not initially available, and eventually expand to include data from other types of collisions, such as those involving lead ions. While the next phase of the collider, Run 3, will require significant updates to the software because the underlying data structure has changed, the current system represents a major milestone. It has successfully demonstrated that it is possible to open up the most complex scientific data in the world without requiring every user to become an expert in the experiment's internal software. By providing a clear, guided path to the data, the LHCb Ntupling Service has turned a massive, static archive into a dynamic resource, allowing the scientific community to explore the universe's secrets with a new level of freedom and ease.