Accelerating discovery across scientific disciplines through reproducible workflows with AiiDAlab

This paper presents AiiDAlab, a Jupyter-based web platform built on the AiiDA infrastructure that empowers scientists across diverse disciplines to automate, manage, and analyze complex computational workflows with full reproducibility, while highlighting its recent expansion into electronic laboratory notebooks, large-scale facilities, and educational settings.

The Gist

Imagine you are a scientist trying to solve a complex puzzle. In the past, doing this required you to be a master mechanic, a data librarian, and a computer programmer all at once. You had to build your own tools, organize your own filing cabinets, and write your own instructions just to run a single experiment. If you made a tiny mistake in the setup, the whole thing would crash, and you'd lose your progress.

AiiDAlab is like a "smart, all-in-one workshop" that changes this experience. Instead of forcing scientists to build their own tools, it provides a pre-furnished, user-friendly interface where they can just click buttons to run complex simulations.

Here is a breakdown of what the paper says, using simple analogies:

1. The Problem: The "DIY" Nightmare

For years, running computer simulations (like modeling how a new battery works or how pollution moves through the air) required deep technical expertise. Scientists had to:

• Write complex code to tell the computer what to do.
• Manually manage thousands of files.
• Keep track of exactly which settings were used for every single test (so they could prove their results later).

It was like trying to bake a cake but having to first invent the oven, mill your own flour, and write the recipe from scratch every time.

2. The Solution: AiiDAlab (The "Smart Workshop")

The authors built AiiDAlab, which sits on top of a powerful engine called AiiDA.

• The Engine (AiiDA): Think of this as the invisible, super-organized robot butler in the background. It automatically remembers every single ingredient you used, every step you took, and every result you got. It ensures that if you do the experiment again, you get the exact same result (reproducibility).
• The Workshop (AiiDAlab): This is the friendly face of the robot butler. It's a website you open in your browser. Instead of writing code, you use simple menus, sliders, and pictures to set up your experiment. It hides the scary technical details so scientists can focus on the science, not the software.

3. How It Works in Different Fields

The paper shows that this "workshop" isn't just for one type of science; it's like a universal adapter that can be plugged into different fields:

• Atmospheric Science (Tracking Pollution): Imagine trying to figure out where pollution comes from by looking at the wind. This requires running millions of tiny simulations. The AiiDAlab app for this (called FLEXPART) lets scientists click a map to select a location, and the system automatically runs the thousands of calculations needed to trace the pollution back to its source.
• Chemistry (Predicting Colors): Scientists want to know how new molecules absorb light (which determines their color). Doing this usually requires a PhD in quantum physics. The AtmoSpec app lets a user type in a chemical name, and the system automatically runs the complex math to predict the color spectrum, showing the results in a simple graph.
• Battery Research (Testing New Batteries): Building and testing batteries is slow and repetitive. The Aurora app connects to a robot that builds and tests batteries. Scientists can design a test plan on the screen, and the robot executes it, while AiiDAlab automatically records every voltage and temperature reading, creating a perfect digital log of the experiment.

4. Connecting the Lab and the Computer

Historically, scientists kept their "lab notebooks" (paper or digital) separate from their computer simulations.

• The Integration: The paper describes a way to link AiiDAlab with Electronic Lab Notebooks (ELNs). It's like having a magic door between your physical lab and your computer. You can send a molecule from your lab notebook to the computer, run a simulation, and have the results automatically pop back into your notebook with all the proof of how they were calculated.

5. Helping Big Facilities and Students

• For Big Science Centers: At places like the Paul Scherrer Institute (which uses giant neutron machines), researchers often struggle to install the right software on their laptops before a short experiment. AiiDAlab acts like a "pre-loaded USB drive" that works instantly. Researchers log in, and they immediately have access to the tools and data they need without installing anything.
• For Students: In classrooms, teachers can give students a link to AiiDAlab. Students can run advanced simulations in minutes without needing to install complex software on their own computers. It teaches them how to do science the "right way" (keeping track of data) from day one.

6. Making It Easy to Start

The authors admit that setting up this system used to be hard. To fix this, they created tools that act like a "one-click installer."

• The Demo Server: They built a public version of the workshop that anyone can try for free. It's like a "test drive" where you can run a small simulation in under a minute to see how it works.
• Local Installation: For those who want to run it on their own computer, they created a tool that sets everything up automatically, so you don't need to be a computer expert to get started.

Summary

In short, AiiDAlab takes the heavy lifting of managing complex computer simulations away from scientists. It turns a chaotic, code-heavy process into a clean, organized, and visual experience. It ensures that every step is recorded automatically, making science more reliable, easier to repeat, and accessible to more people, whether they are in a university classroom or a high-tech research facility.

Technical Summary

Technical Summary: Accelerating Discovery Across Scientific Disciplines through Reproducible Workflows with AiiDAlab

Problem Statement
Despite the increasing computational capabilities available for scientific research, robust and automated workflows remain difficult to orchestrate. While Workflow Management Systems (WFMS) like AiiDA have established core tenets for reproducibility, provenance tracking, and high-throughput execution, the practical execution of these workflows often requires significant technical expertise. Researchers must still configure execution environments, define calculation inputs, interpret complex outputs, and manage the intricacies of parallel code execution on remote High-Performance Computing (HPC) resources. This technical barrier limits the adoption of advanced computational tools, particularly among experimental scientists who wish to focus on research questions rather than infrastructure management. Furthermore, the integration of computational workflows with Electronic Laboratory Notebooks (ELNs) and the handling of large-scale experimental data at facilities remain challenging, often requiring manual data transfer and disparate software stacks.

Methodology
To address these challenges, the authors developed AiiDAlab, a Jupyter-based web platform powered by the AiiDA computational infrastructure. The platform is designed to abstract underlying system complexity through Graphical User Interfaces (GUIs) running in a user's web browser, thereby lowering the cognitive load required to operate sophisticated software systems.

The technical architecture relies on several key components:

• Containerization and Deployment: AiiDAlab is deployed as a containerized application, typically using JupyterHub and Kubernetes (or MicroK8s for single-node setups). This allows for both cloud-based and local deployments, ensuring a consistent, reproducible environment.
• Modular App Ecosystem: The platform utilizes a plugin-based architecture where specific scientific applications (apps) are built as modular extensions. These apps leverage the aiidalab-widget-base library, providing reusable UI components for workflow configuration, job submission, monitoring, and data visualization.
• Provenance and Automation: Under the hood, AiiDA manages the execution of workflows, automatically capturing inputs, outputs, and metadata to ensure full provenance tracking without manual intervention.
• Integration Strategies: The platform integrates with external systems through specific connectors. This includes the aiidalab-eln package for linking with ELNs (demonstrated with openBIS) and custom plugins for hardware control (e.g., the tomato package for battery cyclers) and data parsing (e.g., yadg for experimental data).
• Data Handling: To manage large datasets (e.g., terabytes in climate modeling or gigabytes in neutron scattering), the system employs AiiDA's "stashing" mechanisms to transfer data between temporary scratch spaces and long-term storage, and tools like cubehandler for efficient visualization of volumetric data.

Key Contributions and Applications
The paper details the maturation of AiiDAlab from a materials science tool into a cross-disciplinary platform through several specific implementations:

1. Expansion Beyond Materials Science:

   • Atmospheric Science (FLEXPART App): Developed for the ExAiRIM project, this app automates inverse modeling of greenhouse gas emissions. It manages the generation of concentration footprints using the FLEXPART code and interfaces with the Empa Lagrangian Regional Inversion System (ELRIS) to quantify national emissions. It handles large meteorological datasets by implementing dedicated data-transfer workflows.
   • Atmospheric Chemistry (AtmoSpec App): This application enables the ab-initio prediction of UV/Vis spectra for volatile organic compounds using the Nuclear Ensemble Approach (NEA) and the ORCA code. It guides users from SMILES string input to the generation of Boltzmann-weighted spectra, a task previously restricted to theoretical experts.
   • Battery Research (Aurora App): Integrated with the Aurora robotic platform for battery cell assembly, this app orchestrates charge-discharge cycling experiments. It allows users to design protocols, manage batch submissions for up to 36 cells, and automatically parse experimental data into FAIR-compliant formats using the yadg parser.

2. Integration with ELNs and LIMS:
   The authors demonstrate a "round-trip" integration between AiiDAlab and the openBIS ELN/LIMS. Users can extract molecular inventory data from an ELN, submit simulations via AiiDAlab, and return results (images and provenance archives) to the ELN. The paper proposes the use of semantic annotations (RDF/JSON-LD) to generalize this interoperability across different ELNs and WFMSs, a concept being explored in the PREMISE and MADICES projects.

3. Experimental Data Analysis at Large Facilities:
   At the Paul Scherrer Institute (PSI), AiiDAlab was deployed to support the CAMEA neutron spectrometer and the ICON beamline. The LNS app encapsulates the MJOLNIR analysis code, allowing users to access experimental data "live" via a shared NFS mount within a secure, containerized environment. This eliminates the need for local software installation and manual data transfer, reducing setup time by 1–2 hours per user.

4. Educational Applications:
   AiiDAlab serves as a platform for teaching computational materials science. By providing pre-configured environments (e.g., the Quantum ESPRESSO app with built-in tutorials), it allows students to perform first-principles simulations without installing compilers or managing HPC connections. This approach was successfully tested in a hybrid school at PSI, where participants with little prior DFT experience could launch simulations within minutes.

5. Deployment and Usability Improvements:
   The paper outlines tools to lower adoption barriers, including aiidalab-launch for local Docker-based deployment and standardized Kubernetes Helm charts for institutional multi-user setups. A public demo server (demo.aiidalab.io) is provided for immediate exploration. "Under-the-hood" improvements include automated testing, support for Multi-Factor Authentication (MFA) on HPC resources via the aiidalab-mfa-cscs app, and modern frontend techniques (lazy-loading) to improve responsiveness.

Results
The implementation of AiiDAlab has resulted in:

• Reduced Technical Barriers: Experimentalists and students can now execute complex workflows (e.g., DFT, inverse modeling, battery cycling) via intuitive GUIs without deep domain knowledge of the underlying codes.
• Operational Efficiency: At PSI, the deployment reduced software setup and data transfer times significantly, enabling immediate data analysis during beamtime.
• Reproducibility: The platform ensures that all inputs, processes, and outputs are tracked, allowing for the sharing of simulation states and analysis tools.
• Cross-Disciplinary Adoption: The platform has been successfully adapted for atmospheric science, battery research, and neutron scattering, demonstrating its domain-agnostic nature.
• Educational Impact: In a recent school, over 60% of attendees with no prior DFT experience successfully ran simulations, with high satisfaction ratings (8.5/10).

Significance
The paper positions AiiDAlab not merely as a graphical interface for AiiDA, but as a comprehensive platform for accelerating scientific discovery across disciplines. Its significance lies in its ability to democratize access to high-performance computational resources and complex workflows, shifting routine tasks from specialized computational teams to broader communities of experimental researchers. By integrating with ELNs and large-scale facilities, AiiDAlab bridges the gap between experimental and computational science, fostering a FAIR (Findable, Accessible, Interoperable, Reusable) research environment. The authors emphasize that while the platform originated in materials science, its modular architecture and focus on user experience make it a scalable solution for diverse scientific domains, ultimately aiming to foster open, reproducible, and collaborative research globally.