Enabling stable preservation of ML algorithms in high-energy physics with petrifyML

This paper introduces the petrifyML package, a tool designed to ensure the stable preservation and future reproducibility of high-energy physics machine learning algorithms by converting their configurations into industry-standard ONNX format or native Python and C++ code.

The Gist

Imagine you are a chef who created a revolutionary new recipe for a dish that helps scientists understand the universe. You wrote down the recipe in a very specific, complex notebook that only your current kitchen staff (a specific software version) can read.

Now, imagine that in 10 or 20 years, the kitchen changes. The staff leaves, the software updates, and that specific notebook becomes unreadable gibberish. If someone else wants to cook that dish to verify your results, they can't. They've lost the recipe.

This is the problem scientists in High-Energy Physics (HEP) face with Machine Learning (ML). They use complex "recipes" (algorithms) to analyze data from particle colliders. For a long time, these recipes were just internal tools. But now, the recipes are the results. If the recipes can't be read in the future, the science can't be verified.

Enter petrifyML.

What is petrifyML?

Think of petrifyML as a magical translator and time-capsule machine. Its job is to take those complex, fragile, software-specific recipes and turn them into two things:

1. A Universal Language (ONNX): This is like translating your recipe into a format that every kitchen in the world, past, present, and future, agrees to understand. It's the "PDF" of the machine learning world.
2. Plain English (Native Code): It can also rewrite the recipe into simple, human-readable instructions (C++ or Python code) that don't need any special software to run. It's like writing the recipe on a piece of paper that anyone can read, even if they don't have a computer.

How does it work?

The paper explains that scientists currently use different "kitchen tools" (software packages like TMVA, scikit-learn, lwtnn) to train their models. These tools often speak different dialects or rely on heavy, complicated equipment that might disappear in the future.

petrifyML acts as a bridge:

• The Translator: It takes a model trained in one of these specific tools and converts it into the universal ONNX format. This ensures that even if the original tool vanishes, the model can still be "cooked" (run) using standard, modern tools.
• The Scribe: For simpler models (like Boosted Decision Trees), it doesn't just translate; it rewrites the entire logic into plain text code. This is like taking a complex mechanical watch and drawing out every single gear and spring on paper. You don't need the watch anymore; you just need the drawing to rebuild it. This guarantees the model works exactly the same way forever, without needing any specific software updates.

Why is this important?

The paper highlights a few key benefits:

• No More "It Works on My Machine": Usually, if you try to run an old model on a new computer, it breaks because the software versions don't match. petrifyML removes this dependency.
• Future-Proofing: By converting models to ONNX or plain code, scientists ensure that their work can be re-interpreted decades from now. It's like preserving a document not on a floppy disk (which might rot), but on acid-free paper or a universal digital standard.
• Efficiency: The paper tested this tool and found it works fast and doesn't use much computer memory. The converted files are often smaller than the original ones, making them easy to store and share.

The "Validation" Check

The authors are careful to say: "Just giving you the translated recipe isn't enough; we need to make sure it tastes the same."
So, petrifyML includes a built-in "taste test." When it converts a model, it automatically generates a script that runs the new version and compares it to the old version to ensure they produce the exact same results. If there's even a tiny difference, the user knows something went wrong.

In Summary

petrifyML is a tool designed to save the "recipes" of particle physics from being lost to time. It takes complex, software-dependent machine learning models and turns them into either a universal standard format or simple, human-readable code. This ensures that the scientific discoveries made today can be checked, understood, and trusted by scientists 50 years from now, regardless of what technology exists at that time.

Technical Summary

Technical Summary: Enabling Stable Preservation of ML Algorithms in High-Energy Physics with petrifyML

Problem Statement
Machine learning (ML) in High-Energy Physics (HEP) has evolved from an internal tool for calibration and reconstruction to a central, non-parametric component of physics data analysis. While this shift enhances sensitivity to new physics models, it introduces significant challenges for scientific reproducibility. Current ML algorithms are typically trained and deployed using Python-based tools (e.g., TMVA, scikit-learn, lwtnn) that suffer from version instability, heavy dependencies (particularly the ROOT framework), and format incompatibility.

Existing preservation strategies face limitations:

• Pickle/Joblib files: Highly version-dependent and unstable over time; not suitable for long-term preservation without full containerization.
• ONNX Format: While an industry standard, many HEP-specific tools (TMVA, lwtnn, MVAUtils) do not natively support conversion to ONNX. Furthermore, the long-term stability of ONNX execution environments is not guaranteed without cumbersome containerization.
• Native Code: Converting to human-readable C++ or Python eliminates dependencies but is often limited to small models due to file size constraints.

There is a critical gap in the "chain of algorithm preservation" for converting HEP-specific ML configurations into stable, dependency-free, or industry-standard formats.

Methodology
The authors present petrifyML, a Python package and command-line toolset designed to bridge this gap. The tool converts ML configurations from common HEP frameworks into either the ONNX format or native C++/Python code.

The package is modular, with dependencies installed via pip based on the specific conversion task:

1. Boosted Decision Trees (BDTs):
   • scikit-learn: Converts .pkl or .job files to native C++ and Python.
   • TMVA: Converts XML files (ROOT files are not supported directly for this conversion) to native C++ and Python.
   • MVAUtils: Converts ROOT-based MVAUtils files (originating from xgboost or lgbm) to ONNX. This utilizes the uproot library to parse files without requiring a full ROOT installation.
2. Neural Networks (NNs):
   • TMVA (MLPs): Reads TMVA XML files, reconstructs the architecture and weights in TensorFlow/Keras, and exports to ONNX using tf2onnx.
   • lwtnn: Converts lightweightneuralnetwork JSON files (used in ATLAS triggers) to ONNX. Currently supports a subset of layer types (Dense, Normalization, Softmax) and activation functions (Relu, Sigmoid, Elu, Tanh).

Key Features and Validation

• Metadata Retention: petrifyML attempts to preserve training settings and normalization parameters, though limited by input/output format capabilities.
• Validation Scripts: The tool optionally generates validation scripts that compare the output of the converted model against the original implementation using randomly generated inputs (scaled by the model's cut-value statistics).
• Version Control: For ONNX conversions, users can specify --opset and --ir-version to ensure compatibility with specific OnnxRuntime versions, addressing potential issues with rapidly evolving ONNX standards.
• Native Code Generation: For BDTs, the tool generates human-readable C++ or Python code that is dependency-free, ensuring "verbatim performance in perpetuity" for smaller models.

Results and Benchmarking
The authors benchmarked petrifyML on a suite of 1,230 models (including lwtnn, MVAUtils, scikit-learn, and TMVA models) using an Intel Core i7-14700 CPU.

• Conversion Performance:

  • Memory Usage: Ranges from a few MB for lwtnn/ONNX conversions to ~3.5 GB for large MVAUtils xgboost forests (125,000 trees). Most conversions require less than 200 MB.
  • Time: Conversion times vary significantly. lwtnn to ONNX takes ~0.04s, while large MVAUtils xgboost forests can take >4 minutes. Successive conversions in the same environment are significantly faster due to cached module imports.
  • File Size: Converted files are generally compact. ONNX files are up to 80% smaller than originals (except for highly optimized MVAUtils files, which may increase in size by a factor of 3). Native C++/Python files for TMVA BDTs range from 5,000 to 41,000 lines but remain more space-efficient than original XML formats.

• Inference Performance:

  • Accuracy: Converted ONNX models show relative output errors of less than $10^{-6}$ compared to originals. Native code conversions agree perfectly.
  • Memory: Inference generally requires <100 MB. Native C++ BDTs are significantly more memory-efficient than Python or original implementations.
  • Speed: Inference times are generally small (<0.1s). Native C++ inference for BDTs is often faster than the original model, while Python inference is slower. The relative speed difference is considered negligible given the absolute speed of all methods.

Significance and Claims
The paper positions petrifyML not as a replacement for native export methods when all information is available, but as a necessary solution for preserving models where native export is impossible or where the original training environment is lost.

• Reproducibility: The tool enables the long-term preservation of HEP ML algorithms by converting them into formats (ONNX or native code) that are less dependent on specific toolkit versions or the heavyweight ROOT framework.
• Accessibility: By converting HEP-specific formats (like lwtnn JSON or TMVA XML) to ONNX, the tool allows these models to be used in Python and by reinterpretation frameworks (e.g., Rivet, CheckMATE2) that may not support the original HEP-specific libraries.
• Practicality: The authors claim the tool successfully addresses the "insurmountable issue" of ROOT dependency for many reinterpretation tools and provides a lightweight alternative for preserving large BDT forests that would be impractical to store as plain-text code.

The paper concludes that petrifyML is a practical step toward the "Les Houches guidelines on re-interpretable ML," providing a mechanism to ensure that ML-based experimental studies remain interpretable and reproducible in the long term.