⚡ electrical engineering

A Physics-Informed Digital Twin Framework for Calibrated Sim-to-Real FMCW Radar Occupancy Estimation

This paper proposes a lightweight physics-informed digital twin framework that utilizes calibrated domain randomization to align simulated and real FMCW radar noise statistics, enabling highly accurate occupancy detection and people counting with minimal real-world calibration data.

Original authors: Huy Trinh, Sebastian Ratto, Elliot Creager, George Shaker

Published 2026-01-27

📖 4 min read☕ Coffee break read

CC0 1.0

Original authors: Huy Trinh, Sebastian Ratto, Elliot Creager, George Shaker

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to count people walking down a hallway using a special kind of "seeing" radar. The problem is that teaching a robot using real-world radar data is like trying to teach a child to swim by throwing them into the ocean: it's expensive, dangerous, and you need a lot of help (like hiring people to label every single video frame).

This paper proposes a clever shortcut: teach the robot in a video game first, then let it swim in the real ocean.

Here is how they did it, explained through simple analogies:

1. The "Video Game" Simulator

The researchers built a digital twin (a perfect virtual copy) of a hallway. Inside this video game, they programmed 3D human models to walk around. They also programmed a virtual radar that works exactly like the real one.

The Catch: In the video game, the world is too perfect. The "background noise" (static, echoes off walls) is too clean and quiet. It's like a recording studio with no background hum.
The Result: When they trained a robot brain (a neural network) only on this perfect video game data, the robot failed miserably in the real world. It got confused because the real hallway is noisy and messy, while the game was silent and sterile.

2. The "Static" Problem

Real radar data is full of "static" (noise), like the hiss you hear on an old radio when no station is playing. The video game radar didn't have enough of this hiss.

The Old Way (Random Guessing): Previous methods tried to fix this by randomly adding static to the game data, hoping to get lucky. It's like a chef adding random spices to a soup, hoping one of them tastes like the real dish. It helped a little, but the soup still didn't taste right.

3. The "Calibrated" Solution (The Secret Sauce)

The authors came up with a method called Calibrated Domain Randomization (CDR). Here is the analogy:

The One-Shot Calibration: Before training the robot, they took a tiny, 10-second "sniff" of the real empty hallway. They didn't need to label anything or count people; they just listened to the background hiss of the empty room.
The Match: They used that 10-second sample to create a "noise recipe." They then took their perfect, silent video game data and forced it to have the exact same background hiss and statistical "flavor" as the real hallway.
The Magic: They didn't change the people walking in the game (the important part); they just made the background noise look exactly like the real world.

4. The Results: From Confused to Confident

When they tested the robot trained with this new method:

Occupancy Detection (Is someone there?): The robot went from being basically a coin flip (50% accuracy) to being a master detective (97% accuracy). It could tell the difference between an empty room and a room with people almost perfectly.
People Counting (How many people?): This is harder because two people walking can look similar to one person. The old methods were guessing randomly. The new method got it right 72% of the time, which is a huge jump.

Why This Matters

Think of this like training a pilot. Instead of making them fly a real plane in a storm (expensive and risky), you put them in a flight simulator.

Old Simulators: The wind felt fake. When the pilot got to a real plane, they panicked.
This Paper's Simulator: They measured the wind on the runway, then tweaked the simulator so the wind felt exactly like the real wind. Now, when the pilot steps into the real plane, they feel right at home.

In short: The paper shows that you don't need massive amounts of real-world data to train radar systems. You just need a good simulator and a tiny bit of real-world data to "tune the static" so the simulation feels real. This makes building smart radar systems for counting people much cheaper and faster.

Technical Summary: A Physics-Informed Digital Twin Framework for Calibrated Sim-to-Real FMCW Radar Occupancy Estimation

Problem Statement

Developing robust radar perception models directly from real-world measurements is inherently costly and labor-intensive. It requires controlled experiments, repeated calibration, and extensive manual annotation. While high-fidelity simulators based on full-wave or ray-tracing electromagnetic solvers offer a scalable alternative for generating labeled synthetic data, a significant "sim-to-real" gap persists. Existing approaches often rely on proprietary electromagnetic toolchains, extensive parameter sweeps, or complex generative models (e.g., GANs) that require specific domain knowledge tuning. This paper addresses the challenge of efficiently transferring Frequency Modulated Continuous Wave (FMCW) radar models trained on simulation to real-world occupancy detection and people counting tasks without requiring large labeled real-world datasets.

Methodology

1. Physics-Informed Digital Twin Framework

The authors propose a modular digital twin framework that integrates human animation, electromagnetic modeling, and automated data generation.

Simulation Environment: A corridor environment (walls, ceiling, floor) is reconstructed with measured dielectric properties to capture realistic multipath propagation and clutter.
Radar Modeling: A virtual 60 GHz FMCW radar, modeled after the Infineon BGT60TR13C sensor, replicates the physical hardware's antenna configuration, polarization, and chirp parameters (882.35 MHz bandwidth, 256 samples per chirp).
Human Dynamics: Time-varying radar cross-sections and realistic trajectories are generated using animated 3D humanoid meshes derived from Mixamo and CMU Motion Capture datasets.
Data Generation: An electromagnetic solver using Shooting and Bouncing Rays (SBR) and Physical Optics (PO) methods computes scattered fields. The output is organized into complex data cubes (amplitude and phase) which are converted into Range-Doppler (RD) maps. This baseline simulation is referred to as the RT (Ray-Tracing) baseline.

2. Data Processing Pipeline

Both simulated and real data undergo a standardized preprocessing pipeline:

Transformation: A 2D Fast Fourier Transform (FFT) is applied across fast-time and slow-time dimensions to generate complex RD maps.
Normalization: Magnitudes are converted to decibels (dB), clipped to a fixed range $[v_{min}, v_{max}]$ estimated from simulation, and normalized to $[0, 1]$ .
Visualization: Data is mapped via a perceptually uniform colormap (viridis) to create 3-channel images for input into a ResNet18 model with a task-specific classifier head (2 units for binary occupancy, 3 units for people counting).
Training vs. Testing: Training is performed exclusively on simulated RD maps. A real-world dataset collected in the same corridor (empty, 1 person, 2 people) is held out entirely for evaluation.

3. Calibrated Domain Randomization (CDR)

To bridge the domain gap, the paper introduces a Calibrated Domain Randomization (CDR) step, which aligns the global noise-floor statistics of simulated RD maps with real environments while preserving micro-Doppler structures.

Mechanism: A per-frame, per-cell clamp is applied in the dB domain: $\tilde{S}(i, j) = \max(S(i, j), N(i, j))$ . This operation raises overly clean simulated background cells to a sampled noise-floor level.
Random DR Baseline: Samples noise $N(i, j)$ from a generic Gaussian distribution with heuristic parameters ( $m \in [-130, -100]$ dB, $s \in [1, 4]$ dB).
CDR Approach: Replaces heuristic sampling with a one-shot, data-driven calibration. An unlabeled 10-second (100-frame) sequence of an empty real corridor is used to compute the empirical mean ( $m_t$ ) and standard deviation ( $s_t$ ) of the real noise floor. The simulation then samples noise $N_t(i, j) \sim \mathcal{N}(m_t, s_t^2)$ to drive the calibration.

Key Contributions

Lightweight Sim-to-Real Framework: The paper proposes a framework that enables reliable FMCW radar occupancy detection and people counting using only a physics-informed geometric simulator and a small, unlabeled real calibration set.
Calibrated Domain Randomization (CDR): A novel method that aligns simulated noise-floor statistics to real-world observations. Unlike standard domain randomization, CDR uses a single short real-world recording to calibrate the noise distribution, effectively shifting the simulated histogram toward the real distribution without altering the discriminative micro-Doppler signatures of human motion.
Modular Digital Twin: An integrated system combining electromagnetic solvers (SBR/PO) with motion-capture-driven human animation to generate realistic, labeled RD maps.

Results

The framework was evaluated on a real-world test set for two tasks: binary occupancy detection and people counting (1 vs. 2 people).

Occupancy Detection:
- RT Baseline (Unmodified Simulation): Struggled to generalize, achieving balanced accuracy near 50% (chance level) due to mismatched clutter and noise.
- Random DR: Improved robustness to balanced accuracy of ~83% but still exhibited unrealistic structures.
- CDR: Achieved 97% balanced accuracy, significantly reducing false negatives and false positives compared to baselines.
People Counting:
- RT Baseline: Failed to transfer, achieving ~33% balanced accuracy (random guessing).
- Random DR: Improved performance to ~61% but still heavily confused the three classes.
- CDR: Achieved 72% balanced accuracy, with improved confusion matrices, particularly in distinguishing one vs. two-person cases. Misclassifications were primarily observed in ambiguous scenes with partial overlap or Doppler occlusion.
Feature Analysis: Visualization of ResNet18 activations showed that the CDR-trained model concentrates responses on range-Doppler cells associated with human motion while suppressing background structure, whereas the baseline model produced diffuse responses dominated by simulator-specific artifacts.

Significance and Claims

The paper claims that Calibrated Domain Randomization is a simple, lightweight, and interpretable method for sim-to-real transfer in radar perception. By calibrating simulated noise floors to match the global statistics of unlabeled real measurements, the approach significantly improves model performance on real test data compared to both pure simulation training and uncalibrated domain randomization.

The authors emphasize that this framework is practical for radar-based human sensing systems built from modern simulators, as it avoids the need for proprietary electromagnetic toolchains, extensive parameter sweeps, or large labeled real-world datasets. The work suggests that future research could extend this approach to angle-of-arrival estimation and other activity recognition tasks, but the current contribution is strictly limited to occupancy and counting via noise-floor alignment.