Inpainting physics: self-supervised learning for context-driven fluid simulation

This paper proposes a self-supervised framework that reformulates steady computational fluid dynamics inference as a context-driven inpainting problem, enabling reusable flow priors that outperform traditional supervised models under boundary shifts and support local geometry editing.

The Gist

Imagine you are trying to finish a complex jigsaw puzzle, but you only have a few pieces from the edges and the center. Usually, if you want to solve a fluid dynamics problem (like how blood flows through a brain artery), you act like a traditional engineer: you measure the exact shape of the puzzle box, the size of the pieces, and the rules of the game, then you try to calculate the entire picture from scratch.

This paper proposes a different way to think about the problem. Instead of calculating the whole picture from scratch every time, the authors suggest treating it like filling in a missing part of a drawing (a process called "inpainting").

Here is the breakdown of their idea using simple analogies:

1. The Old Way: The "Recipe" Approach

Traditional computer models for fluid flow are like a chef who memorizes a specific recipe. If you give them the exact ingredients (geometry) and cooking instructions (boundary conditions like how fast blood enters), they can cook the dish (predict the flow).

• The Problem: If you change the ingredients slightly (a different artery shape) or the instructions (a different blood flow speed), the chef gets confused. They can't cook a new dish unless they have practiced that exact combination before. They are rigid and struggle to adapt.

2. The New Way: The "Context-Aware Artist"

The authors suggest training a computer model not as a recipe follower, but as an artist who understands how fluids naturally behave.

• The Training: Instead of showing the model specific recipes, they show it thousands of completed fluid flow pictures. The model learns the "vibe" or the "prior" of how fluids move. It learns that if water is flowing fast on the left, it usually slows down or swirls in a specific way on the right. It learns the rules of the game without being told the specific starting conditions.
• The Inference (The "Inpainting"): When you want to solve a new problem, you don't give the model a recipe. Instead, you give it a blank canvas with a few known pieces fixed in place (like the inlet where blood enters and the outlet where it leaves). You tell the model: "Here are the edges; please fill in the rest based on what you know about how fluids work."

3. The Secret Sauce: "Latent Tokens" (The Shorthand)

Fluid simulations involve millions of data points (like a high-resolution photo). Trying to fill in the missing parts of such a huge image is slow and messy.

• The Analogy: Imagine trying to describe a landscape. Instead of listing the color of every single pixel, you group them into "patches" or "tokens." You say, "This patch is a blue sky," "This patch is a green hill."
• The Paper's Method: They developed a special tool (a "tokeniser") that compresses the massive, messy 3D fluid data into compact, manageable "patches" (tokens). The AI learns to fill in the missing patches. Once it fills them in, the tool expands them back into a full, high-resolution fluid map.

4. Why This is a Big Deal

The paper tested this on blood flow in brain aneurysms (weak spots in arteries).

• Handling Changes: If the traditional model sees a new artery shape or a new blood flow speed it hasn't seen before, it often fails. The new "artist" model, however, just looks at the known parts (the inlet/outlet) and fills in the rest. It handles these changes much better because it learned the general rules of flow, not just specific recipes.
• Editing the Puzzle: Imagine you have a simulation of a blood vessel, and you want to see what happens if the vessel gets slightly wider in one spot.
  • Old Way: You throw away the whole simulation and start over from scratch.
  • New Way: You keep the parts of the simulation that didn't change (the "unchanged context") and only ask the AI to "re-paint" the small area that changed. This is incredibly efficient and accurate.

Summary

The paper argues that instead of training AI to be a calculator that solves equations based on fixed inputs, we should train it to be a creative predictor that understands the physics of flow. By treating fluid simulation as a "fill-in-the-blanks" game where the AI uses the surrounding context to guess the missing parts, the model becomes much more flexible, robust, and capable of handling new, unseen situations.

Key Takeaway: They turned a rigid "input-to-output" calculator into a flexible "context-aware" artist that can fill in missing fluid dynamics based on what it knows about how fluids naturally behave.

Technical Summary

Technical Summary: Inpainting Physics for Context-Driven Fluid Simulation

Problem Statement

Computational Fluid Dynamics (CFD) is essential for engineering and biomedical applications but remains computationally expensive, often requiring the solution of large discretized systems from scratch for every new geometry or boundary condition. Current neural surrogate models typically function as supervised forward operators, mapping explicit problem specifications (geometry, boundary conditions, physical parameters) directly to solution fields (velocity, pressure). While effective within the training distribution, this formulation tightly couples the model to specific conditioning variables. Consequently, these models struggle to generalize to unseen boundary conditions, geometric variations, or shifts in the data-generating process. Furthermore, they cannot easily leverage partial observations or reuse existing simulation data for local edits, as they are designed to predict the entire field from scratch based on scalar inputs.

Methodology

The authors propose reformulating steady CFD inference as a context-conditioned inpainting problem rather than a supervised regression task. Instead of training on explicit boundary condition labels, the model learns a self-supervised prior over velocity fields. Boundary constraints are imposed only during inference by fixing known regions (e.g., inlets, outlets, or unchanged simulation areas) and "inpainting" the unknown regions.

The proposed framework consists of three main stages:

1. Local Neighbourhood Tokeniser:

   • To handle high-resolution 3D point clouds (approx. 200k points) efficiently, the raw velocity field is compressed into compact spatial latent tokens.
   • The method uses farthest point sampling to select 2,500 neighborhood centers. Around each center, 512 nearest points are collected.
   • A PointNet-style encoder maps these local neighborhoods (containing relative coordinates, wall distance, and velocity) into a single latent token ($z \in \mathbb{R}^{256}$).
   • A decoder reconstructs local velocities by broadcasting the latent token and concatenating local geometric descriptors. This preserves spatial locality, which is critical for fixing specific regions during inference.

2. Self-Supervised Generative Models:

   • The models are trained on complete velocity fields without explicit boundary condition labels (e.g., mass flow, inlet profiles). They are "unconditional" regarding simulation parameters.
   • Two backbones are evaluated:
     • Latent Flow Matching (L-FM): A DiT-style transformer trained to predict the velocity of a continuous transport from noise to the data distribution.
     • Latent Masked Autoencoder (L-MAE): A Transformer-based model trained to reconstruct masked latent tokens from visible ones. This aligns directly with the inference task of completing missing regions.

3. Explicit Boundary Inpainting at Inference:

   • Known regions (inlet, outlet, or fixed simulation parts) are encoded into latent tokens and held fixed.
   • The model predicts the missing tokens ($v_U | v_K$).
   • For L-FM, integration starts from noise (or zero) with fixed known tokens. For L-MAE, the masked region is predicted in a single forward pass or iteratively.
   • The system can optionally enforce physical constraints, such as zero divergence (incompressibility), during the integration steps.

Key Contributions

1. Paradigm Shift: The paper proposes viewing CFD emulation as context-conditioned inpainting rather than supervised neural operator modeling. This decouples the learned prior from specific training-time conditioning variables.
2. Improved Generalization: The approach demonstrates significantly better robustness to boundary condition shifts (e.g., unseen mass flows) and dataset shifts (different geometries, solvers, and physical assumptions) compared to supervised baselines.
3. Local Geometry Editing: The framework enables local editing of simulations. By fixing the majority of an existing simulation and only inpainting the modified region, the model can adapt to small geometric perturbations without re-simulating the entire domain.
4. Scalable Architecture: The introduction of a local neighborhood tokeniser allows the application of generative models to large 3D volume meshes, a modality dominant in scientific computing but often difficult for standard grid-based or global pooling approaches.

Experimental Results

The method was evaluated on intracranial aneurysm hemodynamics using the Aneumo dataset (steady-state) and the AneuG dataset (transient, held-out for generalization testing).

• Tokeniser Performance: The local tokeniser achieved low reconstruction error across mass flows, including out-of-distribution (OOD) cases, preserving flow structures better than random downsampling.
• In-Distribution vs. OOD:
  • On in-distribution tasks with sparse context (only inlet/outlet), supervised neural surrogates (e.g., AB-UPT) performed competitively.
  • However, as context increased, the inpainting models (particularly L-MAE) outperformed supervised baselines.
  • Under OOD conditions (mass flows outside the training range, e.g., $m=1$ when trained on $2-3.5$), supervised models collapsed, while L-MAE maintained robust performance by treating the new boundary conditions as observed flow context.
• Dataset Shift: On the external AneuG dataset (different solver, non-Newtonian fluid, different geometries), supervised models degraded significantly. L-MAE was the only model to consistently outperform naive baselines.
• Local Editing: In experiments involving local aneurysm deformations, L-MAE reduced the normalized Mean Squared Error (nMSE) from ~73% (supervised) to ~26% within the masked region. For the full sample, nMSE dropped from ~36% to ~4.5%, demonstrating the ability to reuse unchanged simulation context effectively.

Significance and Claims

The paper claims that viewing CFD inference as context-conditioned inpainting transforms neural surrogates from task-specific predictors into reusable flow priors.

• Flexibility: The model can be queried with varying conditioning patterns, from sparse inlet-outlet constraints to large measured spatial contexts.
• Robustness: By learning a prior over the solution space rather than a mapping from parameters to solutions, the model handles distribution shifts more naturally.
• Efficiency in Editing: The ability to reuse unchanged regions of a simulation offers a practical advantage for design optimization and anatomical studies where only local changes occur.

The authors acknowledge limitations, noting the current focus on steady internal flows and velocity fields only (excluding pressure or wall shear stress). They also note that while local context captures dynamical parameters, structural parameters like Reynolds number or turbulence model choices may require additional conditioning in more general CFD problems. Nevertheless, the work suggests a path toward foundation models for physics that behave less like single-purpose predictors and more like generative priors over physically plausible flow fields.