HS3: A Descriptive, Interoperable Serialization Standard for Statistical Models in High-Energy Physics

This paper introduces HS3, a new implementation-agnostic, human-readable, and extensible serialization standard designed to provide a universal, FAIR-compliant format for representing and exchanging statistical models in high-energy physics, thereby overcoming the limitations of existing software-specific formats.

The Gist

Imagine that high-energy physics (the study of tiny particles like those at the Large Hadron Collider) is like a massive, global cooking competition. For decades, the chefs (scientists) have been creating incredible recipes (statistical models) to explain how the universe works. However, there was a major problem: every chef was writing their recipe in a different, secret language.

Some wrote in "ROOT," a complex code that only specific computers could read. Others wrote in "pyhf," a simpler format that was easy for humans to read but couldn't handle every type of dish. If you wanted to combine two recipes to make a bigger meal, or if you wanted to read a recipe from 10 years ago on a new computer, it was often impossible. The recipes were like handwritten notes on napkins that would rot if the ink faded or the paper got wet.

Enter HS3: The Universal Cookbook

This paper introduces HS3 (High-Energy Physics Statistics Serialization Standard). Think of HS3 as a new, universal language for recipes that solves all these problems.

Here is how it works, using simple analogies:

1. The "Descriptive" Language (The Menu, Not the Chef)

Before, some recipe formats were like giving a robot a list of strict, step-by-step instructions (e.g., "Turn left, then stir for 3 seconds"). If the robot broke, the recipe was useless.
HS3 is different. It is descriptive. It doesn't tell the computer how to cook; it just describes what the dish is. It says, "This is a Gaussian soup," or "This is a Poisson stew."

• The Analogy: Imagine a menu that lists ingredients and flavors but doesn't care if you cook it in a French kitchen, a Japanese kitchen, or a microwave. Because it just describes the dish, any chef (any computer program) can read it and cook it their own way.

2. The "LEGO" Structure (Building Blocks)

The paper explains that HS3 builds models like a computational graph, which is just a fancy way of saying "a map of LEGO blocks."

• You have blocks for ingredients (data).
• You have blocks for rules (functions).
• You have blocks for the final dish (likelihood).
• The Magic: You can snap these blocks together in a clear, logical way. If you want to change one ingredient, you just swap that one block. You don't have to rebuild the whole tower. This makes it easy to see how the whole model is put together, even if it's a giant, complex structure.

3. The "Time-Travel" Feature (Long-Term Preservation)

One of the biggest worries in science is: "Will we be able to read our data in 50 years?"

• Old way: If you save a file in a specific version of a program, and that program disappears in 10 years, your file is a "digital fossil"—unreadable.
• HS3 way: Because HS3 is written in a simple, human-readable format (like JSON, which looks like plain text), it doesn't rely on any specific software. Even if all the current computers break, a human could theoretically read the HS3 file and understand the recipe. It's like writing a recipe in English instead of a secret code; it survives the death of the tools used to write it.

4. The "Translator" (Interoperability)

The paper shows that HS3 acts as a universal translator.

• It can take a recipe written in the old "ROOT" language and translate it into HS3.
• It can take a recipe from "pyhf" and translate it into HS3.
• It can even translate back.
• The Result: A scientist using a Python computer can now share a model with a scientist using a C++ computer, and they can both understand it perfectly. They can even check if they are getting the same results, like two chefs tasting the same soup to make sure the recipe is consistent.

5. Why This Matters Now

The paper argues that the field of physics is moving from just "finding new particles" to "measuring them with extreme precision." This requires combining many different experiments and models together.

• The Problem: You can't combine recipes if they are written in different languages.
• The Solution: HS3 allows scientists to combine these models easily, check them for errors, and publish them so that anyone (even people outside the original team) can use them to test new theories.

Summary

In short, HS3 is a new standard for writing down the "mathematical recipes" of particle physics. It is:

• Human-readable: You can read it with your eyes, not just a machine.
• Universal: It works across different computer languages and software.
• Future-proof: It ensures that today's scientific discoveries can be understood and reused by future generations, regardless of what technology they use.

The paper claims that this standard is already being used to publish data, check results between different computer programs, and even help teach students about statistics. It's the first step toward making the "library of physics" truly open and accessible to everyone.

Technical Summary

Technical Summary: HS3 – A Descriptive, Interoperable Serialization Standard for Statistical Models in High-Energy Physics

Problem Statement
Statistical models in High-Energy Physics (HEP) are the primary scientific artifacts encoding the relationship between observed data, physics parameters, and uncertainties. While likelihood-based inference is central to precision measurements, Effective Field Theory (EFT) fits, and cross-analysis combinations, the current ecosystem for representing these models is fragmented. Existing formats, such as ROOT workspaces, pyhf JSON files, and CMS DataCards, are tied to specific software stacks (e.g., C++/ROOT, Python). This creates significant barriers to interoperability, validation, and long-term preservation. ROOT workspaces are binary and version-dependent; pyhf JSON is human-readable but limited in scope compared to RooFit; and CMS DataCards rely on specific C++ implementations. Consequently, converting between formats is brittle, cross-framework validation is difficult, and the long-term usability of models is threatened by software obsolescence.

Methodology and Design Principles
The authors introduce HS3 (High-Energy Physics Statistics Serialization Standard), an implementation-agnostic, human-readable, and extensible serialization format. The design is guided by several core principles:

• Descriptive Domain-Specific Language (DSL): HS3 is a declarative specification of mathematical structures (distributions, functions, parameters, datasets) rather than an encoding of execution logic. It separates mathematical semantics from algorithmic implementation, allowing different inference engines to interpret the same model.
• Computational Graph Representation: Models are represented as directed acyclic graphs (DAGs) where nodes are named distributions, functions, or data, and edges represent dependencies. The standard supports a hybrid granularity, allowing both high-level composite nodes (e.g., histfactory_dist for template-based models) and low-level elementary operations. This preserves semantic structure while enabling backend optimizations.
• Interoperability and Completeness: HS3 is designed to be semantically complete with respect to RooFit (the most expressive current framework) while retaining the readability of pyhf JSON. It aims to support both binned and unbinned likelihoods, hierarchical composite models, and extended distributions.
• Human and Machine Readability: The canonical format is JSON, chosen for its mature tooling and explicit structure. All objects are uniquely named to ensure unambiguous references, facilitating version control and incremental modification.
• FAIR Principles: The standard aligns with Findable, Accessible, Interoperable, and Reusable principles, utilizing persistent identifiers and open formats to ensure models remain usable as software environments evolve.

Structure of the Standard
An HS3 document is organized into top-level sections containing indexed objects:

• Distributions: Fundamental (e.g., Gaussian, Poisson) and composite distributions.
• Functions: Deterministic mathematical transformations (elementary or user-defined).
• Data: Observed or simulated datasets (binned or unbinned).
• Likelihoods: Objects combining distributions and datasets.
• Domains: Definitions of parameter ranges, types (continuous/discrete), and constraints.
• Parameter Points: Named sets of values for benchmarks or starting points.
• Analyses: High-level prescriptions defining parameters of interest (POIs), nuisance parameters, and test statistics.
• Metadata: Provenance, versioning, and authorship information.
• Misc: Non-normative hints for visualization or configuration.

Key Contributions and Implementations
The paper details the development of HS3 and its integration with existing frameworks:

1. ROOT/RooFit Integration: A built-in library in ROOT 6.36 enables bidirectional conversion between RooWorkspace objects and HS3 JSON, preserving names, parameters, and normalization. This serves as the reference implementation.
2. CMS Combine and DataCards: DataCards can be parsed into RooWorkspaces and subsequently exported to HS3. The modular nature of HS3 allows for future inclusion of custom CMS-specific modeling constructs.
3. pyhf Compatibility: pyhf JSON models can be translated to HS3 by promoting channels to histfactory_dist nodes. Conversely, simple HS3 models can be converted back to pyhf JSON.
4. Cross-Language Implementations:
   • pyhs3: A pure-Python package providing parsers, validators, and tools for constructing HS3 models, ensuring consistency with ROOT.
   • HS3.jl: A Julia implementation offering high-performance parsing and native data structures, facilitating integration with JuliaHEP tools and gradient-based inference.

Results and Early Applications
The paper presents early validation and application of the standard:

• Cross-Validation: Figure 1 demonstrates good agreement between profile likelihood curves generated by the C++ ROOT implementation and the Python pyhs3 implementation for an ATLAS V H measurement, validating numerical consistency across frameworks.
• HEPData Integration: HEPData now supports HS3 via a "badge" system. Several ATLAS analyses (Higgs measurements, new physics searches, SM analyses) have been published with HS3-based likelihoods, allowing external users to reconstruct models without specific software stacks.
• Combined Analyses: Proof-of-concept studies show that multiple public likelihoods can be combined into a single model without manual rewriting, leveraging HS3's explicit encoding of parameter names and constraints.
• DEMOS Project: HS3 has been selected as the reference format for the DEMOS project, which aims to unify statistical modeling across particle physics, astroparticle physics, and nuclear physics.
• Education: The human-readable nature of HS3 makes it suitable for teaching statistical inference, serving as a bridge between conceptual materials and full scientific analyses.

Significance and Claims
The authors posit that HS3 establishes a foundation for FAIR, long-lived statistical models at the LHC and beyond. By decoupling model definitions from specific inference engines and programming languages, HS3 addresses the critical need for reproducibility, reinterpretation, and long-term preservation in an era dominated by systematic uncertainties and complex global fits. The paper claims that HS3 provides a stable, extensible, and framework-independent language that can evolve to meet the needs of the broader scientific community, potentially becoming for statistical models what .root files are for data storage: a durable, portable, and universally exchangeable representation. The standard is not presented as a final, immutable solution but as a living framework governed by community consensus and versioning to ensure backward compatibility and continuous improvement.