⚡ electrical engineering

SynthPix: A lightspeed PIV image generator

SynthPix is a high-performance, JAX-based synthetic image generator for Particle Image Velocimetry (PIV) that streams on-the-fly image pairs directly into learning pipelines to enable efficient data generation, robustness testing, and closed-loop experimental design without the storage overhead of offline workflows.

Original authors: Antonio Terpin, Alan Bonomi, Francesco Banelli, Raffaello D'Andrea

Published 2026-04-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Antonio Terpin, Alan Bonomi, Francesco Banelli, Raffaello D'Andrea

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to drive a boat through a river. To do this, you need to show the robot thousands of videos of water flowing, so it can learn to predict where the water will go next.

In the real world, filming water is expensive, difficult, and sometimes dangerous. You can't just set up a camera in every river on Earth. This is where SynthPix comes in.

Think of SynthPix as a "Magic Water Simulator" that runs at lightspeed. It's a computer program that creates fake, but incredibly realistic, videos of flowing water (specifically for a technique called Particle Image Velocimetry, or PIV) without needing a single drop of real water.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Offline" Bottleneck

Before SynthPix, scientists who wanted to train these AI robots had to play a slow, tedious game:

Step 1: Generate a bunch of fake water videos on a computer.
Step 2: Save them to a hard drive (like filling up a massive warehouse).
Step 3: Load them back into the computer to train the AI.

This was like trying to fill a swimming pool by carrying water in a teaspoon, one spoonful at a time. If you wanted to test how the AI handles different types of rain, wind, or dirty water, you had to build a new warehouse for every single test. It was too slow, too expensive, and too rigid.

2. The Solution: The "Streaming" Factory

SynthPix changes the game by acting like a high-speed conveyor belt factory.

Instead of making videos, saving them, and then loading them, SynthPix streams them directly into the AI's brain while it's learning.
It's built on a special engine called JAX (think of it as a super-charged sports car engine for math) that runs on powerful graphics cards (GPUs).
The Analogy: If old methods were like mailing a letter to your friend, SynthPix is like a live video call where you can see and talk instantly. It generates thousands of image pairs per second.

3. How It Creates "Fake" Water

The program doesn't just guess; it follows the laws of physics to create the illusion of water:

The Particles: Imagine sprinkling tiny, glowing specks of dust into a river. SynthPix places millions of these "specks" on a digital canvas.
The Flow: It takes a mathematical map of how the water is moving (the "flow field") and pushes those specks along the current.
The Camera: It then simulates a camera taking a picture. It adds realistic "imperfections" like blurry spots, uneven lighting, and digital noise, just like a real camera would see.
The Result: You get two images (a "before" and "after" snapshot) that look exactly like a real experiment, complete with the messy details that make real data hard to analyze.

4. Why It's a Game-Changer

The paper highlights three main superpowers of SynthPix:

Speed: It is orders of magnitude faster than anything else. While other tools might generate a few images a minute, SynthPix can generate hundreds of thousands in the same time. It's the difference between a snail and a supersonic jet.
Flexibility: Because it streams data, scientists can change the rules while the AI is learning. Want to see how the AI handles a sudden storm? Just flip a switch, and the next batch of images will be stormy. No need to stop and rebuild the warehouse.
Reliability: The authors tested it by feeding the fake images into standard AI tools. The AI performed just as well on the fake images as it did on real ones. This proves the "fake" water is good enough to train real-world experts.

The Bottom Line

SynthPix is a tool that lets scientists and engineers generate infinite amounts of high-quality, realistic water-flow data instantly. It removes the bottlenecks of storage and speed, allowing the next generation of AI to learn faster, smarter, and more robustly.

Instead of waiting years to collect enough data to train a robot to understand rivers, SynthPix lets them do it in days—or even hours. It's a "lightspeed" generator that is helping to revolutionize how we measure and understand the flow of fluids in our world.

1. Problem Statement

Particle Image Velocimetry (PIV) is a standard method for measuring fluid flow fields, but the development of modern, data-hungry deep learning estimators is hindered by the limitations of existing synthetic image generation workflows.

Offline Generation Bottlenecks: Traditional synthetic generators create datasets offline. As flow simulation databases grow (e.g., Johns Hopkins Turbulence Database with >200 TB), generating, storing, and transferring terabytes of synthetic image pairs becomes a prohibitive bottleneck for training.
Rigidity: Offline workflows lack flexibility. Changing flow sources, rendering parameters (e.g., noise, seeding density), or acquisition conditions requires regenerating the entire dataset, making iterative processes like hard-example mining, adaptive sampling, and parameter co-design inefficient.
Lack of Integration: Existing tools (often written in MATLAB or Python/NumPy) do not integrate seamlessly with modern machine learning pipelines (PyTorch/JAX) and lack support for batched generation on hardware accelerators (GPUs).

2. Methodology

SynthPix is an open-source Python package implemented in JAX to leverage hardware acceleration (GPUs/TPUs) for high-throughput, on-the-fly image generation.

Architecture & Pipeline:
- Input: Accepts ground-truth flow fields ( $u(x,y), v(x,y)$ ) from various sources (HDF5, MAT, NPY, or procedural Python functions).
- Particle Sampling: Allocates particles based on a configurable "particles-per-pixel" (ppp) density. For the second image in a pair, particles are displaced according to the flow field mapping: $(x, y) \to (x + u\Delta t, y + v\Delta t)$ .
- Mathematical Model: Models particle intensity using a 2D Gaussian distribution:
  $I(x, y) = I_0 \exp\left(-\frac{1}{2(1-\rho^2)} \left[ \frac{(x-x_0)^2}{\sigma_x^2} - \frac{2\rho(x-x_0)(y-y_0)}{\sigma_x\sigma_y} + \frac{(y-y_0)^2}{\sigma_y^2} \right]\right)$
  Parameters like peak intensity ( $I_0$ ), particle diameter, and aspect ratio are sampled from configurable distributions. The model supports axis-misaligned particles ( $\rho > 0$ ).
- Rasterization: Uses parallelized Gaussian kernel generation and summation to render images. To maintain constant tensor shapes for GPU efficiency, "hidden" particles (those masked out) are rendered with zero intensity rather than being removed from the array.
- Streaming: Designed to stream image pairs directly into training loops or benchmarking pipelines, eliminating the need for intermediate storage.
Configuration: Users define parameters via a YAML file, controlling image shape, batch size, flow field reuse, seeding density ranges, noise levels, blur, and illumination non-uniformity.

3. Key Contributions

High-Throughput Generation: SynthPix achieves throughput orders of magnitude higher than existing tools (e.g., PIVlab, Synpivimage) by utilizing JAX's just-in-time (JIT) compilation and GPU parallelism.
Pipeline Integration: It is the first PIV generator designed to integrate directly with deep learning frameworks (PyTorch/JAX), supporting batched generation and on-the-fly data streaming.
Flexibility & Extensibility: It supports diverse flow field sources (DNS, experimental data, analytical flows) and allows for systematic sweeps over nuisance factors (noise, blur, seeding) without regenerating datasets.
Open-Source Ecosystem: The package includes comprehensive documentation, tutorials, unit tests, and follows open-source best practices (MIT License), addressing a gap in reproducibility for fluid dynamics research.

4. Results

The authors evaluated SynthPix on an Ubuntu machine with an AMD Threadripper PRO and four Nvidia RTX 4090 GPUs.

Throughput Performance:
- SynthPix outperforms all existing alternatives by orders of magnitude.
- On a single GPU with a batch size of 256, it generates ~10,000+ image pairs per second (depending on resolution), whereas CPU-based or non-batched methods generate significantly fewer.
- Throughput scales linearly with the number of GPUs.
Image Quality & Validity:
- Self-Consistency: When re-rendering experimental PIV data (2nd PIV Challenge) and re-estimating flow, the error (End-Point Error, EPE) remained low ( $C_{avg} = 0.536$ ), confirming that SynthPix preserves flow information.
- Benchmark Prediction: When used to evaluate the DIS optical-flow estimator against ground truth, the relative EPE discrepancy between original and SynthPix-generated images was negligible (average 0.001).
- Training Efficacy: A RAFT-based estimator trained only on SynthPix-generated data achieved a competitive test EPE of 0.065 on original experimental images, proving the synthetic data is sufficient for training robust models.
Ablation Studies:
- Throughput decreases linearly with increased seeding density and image size but remains high even for large resolutions (up to 2048x2048).
- Batch size significantly boosts throughput initially (amortizing overhead) before plateauing.

5. Significance

SynthPix represents a paradigm shift in how synthetic data is utilized for fluid dynamics and computer vision.

Enabling Data-Driven Methods: It removes storage and I/O bottlenecks, allowing researchers to train massive neural networks on synthetic data that would otherwise be impossible to store or generate.
Adaptive Workflows: It facilitates closed-loop procedures, such as adaptive sampling and acquisition-parameter co-design, where data generation must dynamically respond to model performance.
Standardization: By providing a unified, reproducible interface for PIV image generation, SynthPix reduces the "reinvention of the wheel" in research, fostering better benchmarking and comparison across the community.
Future Outlook: The authors plan to extend the tool to support volumetric (tomographic) PIV, non-orthogonal camera models, and multi-frame particle tracking, further bridging the gap between simulation and real-world fluid measurement.