TransportBench: A Comprehensive Benchmark for Non-Equilibrium Flow Transport

This paper introduces TransportBench, a comprehensive high-fidelity dataset and standardized benchmark designed to evaluate and diagnose scientific machine learning models across diverse non-equilibrium flow regimes, revealing that no single neural architecture universally outperforms others and that specific inductive biases are required for different flow characteristics.

The Gist

Imagine you are trying to teach a robot how to predict how air moves around objects. For years, scientists have mostly taught robots using "smooth" scenarios, like wind blowing gently over a car or water flowing in a pipe. These are predictable, calm situations.

But in the real world, things get chaotic. Think of a rocket re-entering the atmosphere at hypersonic speeds (where the air gets super hot and acts weirdly) or air flowing through a tiny microchip (where the air is so thin it acts more like individual bouncing balls than a smooth fluid). In these extreme situations, the usual rules of physics break down, and the air behaves in "non-equilibrium" ways—meaning it's out of balance, full of sharp shocks, and unpredictable.

The Problem:
Until now, there was no good "driving school" for AI to learn these chaotic, extreme conditions. Existing tests were like driving on a calm, empty highway. They didn't test if the AI could handle a sudden tornado, a jagged rock, or a microscopic maze. Without a proper test, we didn't know which AI models were actually smart enough to handle real-world chaos.

The Solution: TransportBench
The authors created TransportBench, which is essentially a "chaos gym" for AI models. It's a massive collection of high-quality data and a standardized set of tests designed specifically to break AI models and see how they recover.

Think of it like a video game with four distinct levels, each designed to test a different skill:

1. Level 1: The Shape-Shifter (Airfoil Task)

   • The Challenge: The AI must predict how air flows around airplane wings that keep changing their shape.
   • The Test: Can the AI learn the rules of aerodynamics so well that it can guess the outcome for a wing shape it has never seen before?
   • The Result: Models that are good at looking at grids and local patterns (like U-Net) did the best. They were like artists who could quickly sketch a new wing shape and immediately know how the wind would wrap around it.

2. Level 2: The Speed Demon (Cylinder Task)

   • The Challenge: Predicting air flow around a cylinder, but this time the speed and density of the air change wildly.
   • The Test: Can the AI handle a situation where the wind goes from a gentle breeze to a supersonic roar, changing the entire shape of the wake behind the object?
   • The Result: Again, models with strong "local" vision (U-Net) won. They were good at seeing how the immediate surroundings changed as the speed increased.

3. Level 3: The Microscope (Cavity Task)

   • The Challenge: This is a "zoom-in" test. Instead of just looking at the big picture (wind speed), the AI has to predict the behavior of individual gas particles and their hidden statistics.
   • The Test: Can the AI understand the microscopic dance of particles, not just the macroscopic flow?
   • The Result: A model called Point Transformer (which looks at points individually rather than a grid) won. It was like having a detective who could track every single suspect in a crowd, rather than just looking at the crowd as a whole.

4. Level 4: The Shockwave (Double-Cone Task)

   • The Challenge: This is the hardest level. It involves a rocket cone moving so fast it creates massive, sharp shockwaves and chemical reactions. The data is sparse (few examples) and the changes are violent.
   • The Test: Can the AI draw a sharp, jagged line without blurring it? Can it handle the "explosive" parts of the data?
   • The Result: This was a tie-breaker.
     • U-Net was best at getting the exact numbers right (low error in absolute terms). It was like a surgeon who made precise cuts.
     • FNO (a model that looks at the whole picture at once) was best at getting the overall shape right relative to the size of the shock.
     • The Twist: The authors tried adding "high-frequency" features (giving the AI extra tools to see sharp details). For some models, this helped; for others, it made the picture "jittery" with noise. It proved that there is no "one-size-fits-all" tool.

The Big Takeaway
The paper's main conclusion is simple: There is no "perfect" AI model for everything.

• If you need to predict how a new wing shape affects wind, use a grid-based model (like U-Net).
• If you need to track individual particles, use a point-based model (like Point Transformer).
• If you are dealing with violent shockwaves, you have to be careful about which tools you use, because some tools smooth things out too much, while others make them too noisy.

Why This Matters
TransportBench isn't just a list of scores; it's a diagnostic tool. It tells scientists, "Hey, your model is great at smooth curves but terrible at sharp edges," or "Your model is good at the big picture but misses the tiny details."

By providing this standardized "chaos gym," the authors hope to stop researchers from just guessing which AI model to use. Instead, they can now pick the right tool for the specific type of extreme physics they are trying to simulate, whether it's designing a hypersonic jet or understanding gas flow in a microchip.

In short: The paper built a rigorous testing ground to show that in the world of extreme physics, different AI models have different superpowers, and you have to choose the right one for the job.

Technical Summary

Technical Summary of TransportBench: A Comprehensive Benchmark for Non-Equilibrium Flow Transport

Problem Statement
Scientific machine learning (SciML) is increasingly transforming fluid mechanics research; however, existing datasets and benchmarks (e.g., PDEBench, FlowBench) are primarily limited to continuum fluids near thermodynamic equilibrium. These benchmarks typically feature smooth flow fields, low-order macroscopic variables, and regular domains. They fail to capture the defining challenges of non-equilibrium transport, such as rarefaction effects, Knudsen layers, high-order moment quantities, strong shock discontinuities, and multi-scale kinetic-to-continuum behavior. Consequently, high performance on continuum benchmarks does not guarantee robustness in predicting rarefied or hypersonic non-equilibrium flows. Furthermore, existing evaluations often lack standardized protocols, making it difficult to distinguish the impact of architectural inductive biases from differences in parameter budgets, grid resolutions, or training strategies.

Methodology
The authors introduce TransportBench, a high-fidelity dataset and standardized benchmark designed to evaluate SciML models across diverse non-equilibrium flow regimes. The framework is built upon a unified physical formulation based on statistical mechanics, ranging from the Boltzmann equation to macroscopic conservation laws.

• Dataset Construction: The dataset encompasses four representative flow scenarios generated using high-fidelity solvers (Direct Simulation Monte Carlo for rarefied flows, Discrete Velocity Method for kinetic moments, and state-to-state thermochemical CFD for hypersonic flows):

  1. Airfoil Flow (Geometry-Dependent): Rarefied flow over RAE2822 airfoils with geometric variations (CST perturbation) to test generalization to unseen shapes.
  2. Cylinder Flow (Parameter-Dependent): Flow around a fixed cylinder across a wide range of Mach ($Ma$) and Knudsen ($Kn$) numbers to test generalization to operating conditions.
  3. Lid-Driven Cavity (High-Order Kinetic): Prediction of particle distribution functions and high-order moments (stress tensor, heat flux) to test micro-macro connections.
  4. Double-Cone Flow (Shock-Dominated): High-enthalpy hypersonic flow with thermochemical non-equilibrium, strong shocks, and sparse, anisotropic data to test shock resolution.

• Unified Learning Formulation: All tasks are framed as input-output mappings ($G: A \to U$), where inputs include geometry and physical parameters, and outputs include macroscopic variables and non-equilibrium quantities (e.g., distribution functions, stress).

• Benchmarking Protocols: The study evaluates six representative neural architectures (U-Net, Convolutional Autoencoder, DeepONet, Fourier Neural Operator, Vision Transformer, and Point Transformer) under controlled settings. Key design choices include:

  • Parameter Budgets: Fixed to ~1M parameters for Tasks I-III and ~33M for the data-limited Task IV to ensure fair comparison.
  • Preprocessing: Unified grid mapping, binary geometry masking (to exclude solid regions), and logarithmic dynamic-range compression for variables with large variations.
  • Ablation: Evaluation of Fourier feature injection to diagnose spectral bias and shock resolution capabilities.
  • Metrics: Masked Mean Squared Error (MSE), Mean Absolute Error (MAE), and Relative $L_2$ error (computed in physical space for shock tasks to avoid underestimating peak errors).

Key Contributions

1. High-Fidelity Non-Equilibrium Dataset: A comprehensive dataset covering continuum and rarefied regimes, low-speed and hypersonic flows, inert and reacting gases, and both translational and internal-energy non-equilibrium.
2. Standardized Evaluation Framework: A unified protocol that isolates architectural inductive biases from implementation details, enabling systematic comparison across different flow regimes.
3. Diagnostic Tasks: Specific tasks designed to probe distinct challenges: geometric generalization, parameter generalization, high-order kinetic prediction, and shock-dominated reconstruction.
4. Ablation on High-Frequency Injection: A controlled study on the effects of explicit high-frequency feature injection in shock-dominated flows.

Numerical Results
The experiments reveal that model performance is strongly regime-dependent; no single architecture consistently outperforms others across all tasks:

• Geometry-Dependent (Airfoil): Convolutional models (U-Net, Autoencoder) and Vision Transformers performed best, suggesting structured-grid priors are effective for mapping shape variations to shock/wake structures.
• Parameter-Dependent (Cylinder): U-Net achieved the lowest errors, indicating that local convolutional priors effectively capture parameter-induced topological changes in shock and wake structures.
• High-Order Kinetic (Cavity): Point Transformer achieved the lowest error, followed by Vision Transformer, suggesting that flexible point-based aggregation and token-level interactions are well-suited for smooth but physically coupled kinetic fields.
• Shock-Dominated (Double-Cone):
  • Local Priors: U-Net (without Fourier features) achieved the lowest absolute errors (MAE/MSE), highlighting the value of local convolutional priors for resolving sharp gradients.
  • Spectral Bias: Coordinate-based models (DeepONet) tended to smooth shock peaks, while spectral models (FNO) exhibited oscillatory artifacts near discontinuities.
  • Fourier Feature Injection: Explicit high-frequency injection reduced Relative $L_2$ errors for all architectures in the shock-dominated task but introduced a trade-off: for U-Net and Autoencoders, it improved global field agreement (Relative $L_2$) while slightly increasing absolute errors (MAE/MSE) due to background noise.

Significance and Claims
The authors claim that TransportBench serves as a necessary diagnostic testbed for developing SciML methods beyond the Navier-Stokes level. The benchmark demonstrates that:

• Inductive Bias Matters: The suitability of a neural architecture depends on the dominant physical structure of the problem (e.g., local gradients vs. global correlations vs. sharp discontinuities).
• Capacity is Not a Panacea: Increasing model capacity alone does not overcome the difficulties of non-equilibrium prediction; architectural alignment with physical phenomena (e.g., locality for shocks, flexibility for kinetic coupling) is critical.
• Evaluation Must Be Multi-Faceted: Single aggregate metrics are insufficient. Accurate assessment requires considering multiple metrics (absolute vs. relative error) and qualitative physical behavior, especially when dealing with high-frequency features and shock discontinuities.

TransportBench is presented not as a leaderboard to crown a single "best" model, but as a tool to identify which inductive biases are appropriate for specific non-equilibrium transport regimes, thereby guiding the development of more robust, physics-aware, and regime-adaptive neural solvers.