TrackCore-F: Deploying Transformer-Based Subatomic Particle Tracking on FPGAs

This paper presents methodologies and tools for deploying Transformer-based subatomic particle tracking models onto FPGAs, addressing current tool limitations and resource constraints through monolithic or automated partitioned synthesis to achieve low-latency inference.

The Gist

Imagine you are trying to solve a massive, chaotic puzzle in the middle of a hurricane. This is what happens at the Large Hadron Collider (LHC), the world's biggest particle accelerator. Every second, it smashes particles together, creating thousands of tiny "breadcrumbs" (called hits) that fly in all directions. Scientists need to connect these dots to figure out what kind of particle was created. This process is called tracking.

For years, this was done by powerful computers in a lab after the experiment was over (like solving the puzzle the next day). But with the upcoming upgrades to the LHC, there will be so much data that scientists need to solve the puzzle instantly, while the particles are still flying.

Here is a simple breakdown of what the paper "TrackCore-F" is trying to do:

1. The Problem: Too Smart, Too Slow

Scientists have developed a very smart type of computer brain called a Transformer (the same technology behind AI chatbots) to solve these particle puzzles. It's incredibly accurate.

However, these "brains" are heavy. They usually need giant, power-hungry graphics cards (GPUs) to run, which are too big and slow for the actual particle detectors. The detectors need a solution that is:

• Tiny: Small enough to fit on a chip.
• Fast: Instantly processing data.
• Efficient: Using very little electricity.

2. The Solution: The "Swiss Army Knife" Chip (FPGA)

The authors propose using a special chip called an FPGA (Field-Programmable Gate Array).

• The Analogy: Think of a standard computer chip (like in your phone) as a pre-made kitchen. You can cook, but you can only use the tools that are already built-in.
• The FPGA: Think of this as a Lego workshop. You can build any tool you need, specifically designed for the exact job at hand. If you need a specific shape to catch a particle, you build that shape right into the chip.

3. The Challenge: Fitting a Whale in a Fishbowl

The Transformer models are like whales. The FPGA chips are like fishbowls. You can't just shove the whole whale inside.

• The Strategy: The team figured out how to cut the whale into manageable slices. They take the most important part of the AI brain (the "encoder" layer) and build a custom Lego tool for just that slice.
• The Workflow: They take the AI model, chop it up, and turn the middle slice into a custom hardware circuit. The rest of the model runs on the regular computer part of the chip, while the "heavy lifting" happens on the custom Lego tools.

4. The Trade-off: Speed vs. Accuracy (The "Blurry Photo" Effect)

To make the chip faster and smaller, you usually have to simplify the math. This is called quantization.

• The Analogy: Imagine taking a high-definition photo and turning it into a pixelated, low-resolution image to make it load faster.
• The Result: The paper found that if you simplify the math too much (turning the "photo" into a very blurry sketch), the AI starts making mistakes. It's like trying to identify a friend in a crowd when they are wearing a mask and the lights are dim. They found that keeping the "activations" (the active thinking parts of the AI) precise is crucial, even if you simplify the "weights" (the memory parts).

5. The Bottleneck: The "Memory Closet"

Even with the Lego approach, space is tight.

• The Analogy: Imagine you are building a house, but you only have a small closet for your tools.
• The Finding: The chip has plenty of "logic" (tools to do math), but very little "memory" (space to hold the data while working). Their current design uses about 38% of the available memory closet. This means they can only fit about 4 layers of the AI brain onto this specific chip before the closet is full.

The Bottom Line

This paper is a blueprint for how to shrink a giant, complex AI brain down to fit inside a tiny, super-fast chip that can sit right next to a particle detector.

• Why it matters: It allows scientists to see what's happening in the universe in real-time rather than waiting days for results.
• The Catch: It's a delicate balancing act. You have to cut the AI down to size without making it "dumb," and you have to manage the tiny amount of memory space available on the chip.

They haven't solved the whole puzzle yet, but they've successfully built the first custom tools to start fitting the pieces together.

Technical Summary

1. Problem Statement

The paper addresses the critical challenge of deploying Transformer-based Machine Learning (ML) models for subatomic particle tracking in High-Energy Physics (HEP) experiments, specifically for the upcoming High-Luminosity LHC (HL-LHC).

• Latency Constraints: Traditional tracking is a "post-mortem" (offline) task due to high computational costs. Future experiments require online or pseudo-online tracking to handle increased data rates, necessitating ultra-low latency.
• Hardware Limitations: While GPUs dominate ML deployment, they are often unsuitable for on-site, low-latency, energy-efficient HEP triggers. FPGAs are a viable alternative but face significant constraints regarding resources (BRAM, LUTs, DSPs) and tooling support.
• Tooling Gaps: Current tools for synthesizing Transformer models onto FPGAs are limited or non-existent. Large models cannot be deployed directly and require automated or meaningful partitioning strategies, which are currently lacking.
• Accuracy vs. Efficiency: Aggressive optimization techniques like quantization (reducing precision to INT8) often degrade model accuracy, creating a trade-off between resource usage and prediction reliability.

2. Methodology

The authors propose a structured development flow to deploy Transformer models on FPGAs, utilizing the TrackFormers project models.

A. Target Models

The study focuses on two Encoder-only Transformer architectures designed for "single-shot" tracking (processing an entire event at once):

1. EncCla (Encoder-Classifier): Predicts hit-to-track associations via pre-defined class labels (binning track parameters).
   • Scale: ~1.5M parameters.
2. EncReg (Encoder-Regressor): Regresses track parameters directly, requiring post-processing clustering (HDBSCAN).
   • Scale: ~76k parameters.

B. Hardware Platform

• Device: ARM Zynq UltraScale+ MPSoC ZCU102 (ZU9EG).
• Specs: Quad-Core Cortex-A53 (Processing Unit) + Programmable Logic (PL) with ~599k Logic Cells, 2,520 DSP Slices, and 32.1 MB Block RAM.

C. Development Flow & Toolchain

The authors established a manual partitioning workflow using the following toolchain:

1. PyTorch: Pre-trained models.
2. ONNX: Conversion to Open Neural Network Exchange format to expose low-level operations and dimensions.
3. AMD Vitis HLS 2022.2: Used to write C/C++ kernels for specific model slices (e.g., a single encoder layer) and synthesize them into RTL.
4. AMD Vivado 2022.2: Integrates the HLS-generated IP with the MPSoC block, manages AXI4 communication, and generates the final bitstream.
5. PYNQ: Used for Python-based interaction with the FPGA.

Key Implementation Strategy:
Instead of implementing every ONNX operation individually, the team developed monolithic kernels for specific model segments (e.g., a full encoder layer including Multi-Head Attention, Add & Norm, and Feed-Forward layers). This approach reduces resource overhead compared to granular operation mapping. The workflow involves splitting the model into three parts: pre-slice, the FPGA-synthesized slice, and post-slice, with data flowing between them.

D. Optimization Techniques

• Quantization: Investigated the impact of reducing weights and activations from INT16/FP32 to INT8.
• Partitioning: Manual splitting of models to fit within FPGA resource limits, focusing on synthesizing single encoder layers as a proof of concept.

3. Key Results

A. Quantization Impact on Accuracy

The study found that quantization significantly impacts model performance, particularly when applied to activations.

• Baseline Accuracy: 0.97 (Full precision).
• INT8 Weights + INT16 Activations: 0.71 accuracy (Significant drop).
• INT16 Weights + INT8 Activations: 0.90 accuracy.
• INT8 Weights + INT8 Activations: 0.70 accuracy.
• Conclusion: Quantizing activations has a more detrimental effect on accuracy than quantizing weights. The current toolchain (ONNX Static Quantization) does not easily support "Quantized Weights + Full Precision Activations," limiting optimization options.

B. Resource Utilization (Single Encoder Layer)

Using the ZCU102 FPGA, the authors synthesized a single encoder layer kernel:

• Block RAM (BRAM): 347/912 used (38.04%). This is the primary bottleneck.
• LUTs: 65,815/274,080 used (24.01%).
• DSPs: 67/2,520 used (2.66%).
• Flip-Flops: 74,184/548,160 used (13.53%).

Scalability Estimate: Since BRAM is the limiting factor at ~38% for one layer, the authors estimate that up to 4 encoder layers could theoretically be implemented on this specific FPGA for this model, provided memory transfer strategies (e.g., using DDR instead of on-chip BRAM) are optimized to reduce BRAM pressure.

4. Key Contributions

1. Deployment Workflow: A validated, end-to-end methodology for converting PyTorch Transformers to FPGA kernels using Vitis HLS and Vivado, specifically tailored for HEP tracking tasks.
2. Monolithic Kernel Approach: Demonstrated that grouping operations into monolithic kernels (rather than individual ONNX ops) improves resource efficiency (CLB and memory usage).
3. Quantization Analysis: Provided empirical data showing that activation quantization is the primary driver of accuracy loss in Transformer tracking models, warning against naive INT8 deployment.
4. Feasibility Study: Proved that partial deployment (synthesizing specific layers) is a viable strategy for running complex ML models on resource-constrained FPGAs, enabling on-site inference.

5. Significance

• Enabling Online Tracking: This work moves HEP tracking from offline post-processing toward real-time, on-detector decision-making, which is crucial for the HL-LHC era where data rates will overwhelm current offline capabilities.
• Hardware Efficiency: By targeting FPGAs, the approach offers superior energy efficiency and lower latency compared to GPU-based solutions, making it suitable for embedded, on-site deployment.
• Tooling Advancement: The paper highlights the current gaps in automated FPGA synthesis tools for Transformers and provides a practical, manual workaround that can guide future tool development.
• Model-Accuracy Trade-off: The findings on quantization serve as a critical guideline for future HEP ML deployments, suggesting that maintaining high-precision activations may be necessary to preserve the physics performance of Transformer models.