🔬 applied physics

Physics-guided machine learning for sim-to-real calibration of NV diamond magnetometers

This paper introduces a physics-guided hybrid machine learning framework that embeds Zeeman splitting into the learning pipeline to overcome simulation-to-reality mismatches and data scarcity, achieving a 372-fold precision improvement in calibrating NV diamond magnetometers for robust, self-calibrated vector magnetometry.

Original authors: Jonathan Daniel, Martin Y. Kim, Jesse Hernandez, Emanuel Suarez, Sangwoo Lee, Jinhee Lee, Je-Hyung Kim

Published 2026-08-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jonathan Daniel, Martin Y. Kim, Jesse Hernandez, Emanuel Suarez, Sangwoo Lee, Jinhee Lee, Je-Hyung Kim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where the invisible forces of magnetism could be mapped with the precision of a microscope, even in the chaotic, unshielded environments of the real world. For decades, scientists have relied on a specific type of defect inside a diamond crystal to act as a tiny, ultra-sensitive compass. These defects, known as nitrogen-vacancy centers, are essentially missing atoms in the diamond lattice that behave like quantum sensors. When you shine a green laser on them and send in microwaves, they glow with a brightness that changes depending on the strength of the magnetic field around them. This glowing response, called optically detected magnetic resonance, allows researchers to measure magnetic fields with incredible sensitivity, down to the level of individual atoms. However, turning this delicate quantum phenomenon into a rugged, field-ready tool has been a stubborn challenge. In a quiet laboratory, where temperature is stable and noise is absent, these sensors work beautifully. But step outside into a world of fluctuating temperatures, shifting electrical interference, and imperfect equipment, and the signal becomes a tangled mess. Traditional methods for untangling this signal often require massive amounts of data or constant connection to external references like GPS, which defeats the purpose of having a self-contained sensor.

A team of researchers from California State University, San Bernardino, and the Ulsan National Institute for Science and Technology in South Korea has found a way to teach computers how to read these messy signals without needing a perfect environment or a mountain of data. They developed a new kind of artificial intelligence that does not just guess patterns from numbers but is forced to understand the actual laws of physics that govern the diamond sensors. Instead of letting the computer learn purely by trial and error, which often leads to confusion when the data is noisy, the researchers built the fundamental rule of how magnetic fields split energy levels directly into the learning process. This approach, which they call physics-guided machine learning, acts like a strict teacher who ensures the student never forgets the basic rules of the universe, even when the test questions are difficult.

The researchers tested three different ways to train a computer to decode the magnetic field from the diamond's glow. The first method was a standard statistical approach, where the computer simply looked at the raw data and tried to find a pattern without any help from physics. This method struggled immensely, especially when the magnetic fields were weak, often producing wildly inaccurate results that swung back and forth. The second method embedded the physical laws of the diamond directly into the training. By forcing the computer to respect the specific relationship between the magnetic field and the frequency of the light, the model became much more stable. It learned to ignore the noise and focus on the true signal. The third and most successful method combined the first two into a single, powerful system. This hybrid approach took the stability of the physics-based model and refined it with the flexibility of the statistical model, creating a tool that could handle the imperfections of real-world hardware.

The results of this experiment were striking. When the researchers tested their new system against the old, purely statistical method, the improvement was dramatic. The new physics-guided model reduced the average error by nearly 99 percent, achieving a level of precision that was 372 times better than the baseline. In practical terms, this means the sensor could determine the strength of a magnetic field with a margin of error of just 51 billionths of a Tesla on simulated data, compared to the much larger errors of the previous methods. Crucially, this high-precision metric applies to the system's performance on synthetic data. When the system was deployed to decode real, uncalibrated experimental data, it still successfully bridged the gap between simulation and reality, achieving a mean absolute error of 7.97 Gauss. This demonstrated that the system could learn to recognize the specific quirks and imperfections of their actual equipment, allowing it to decode raw, noisy signals with exceptional reliability.

This work suggests a new path forward for quantum sensing technology. It demonstrates that by combining the speed and adaptability of artificial intelligence with the unbreakable constraints of physical laws, scientists can create sensors that are both highly accurate and robust enough for the real world. The researchers showed that their method could successfully decode uncalibrated data from a diamond sensor, a capability that could eventually allow these devices to operate independently in remote locations, on moving vehicles, or in environments where GPS signals are unavailable. By proving that a machine learning model can be guided by the fundamental equations of nature, the team has opened the door to a new generation of self-calibrating instruments. These tools could one day map the Earth's magnetic field from a drone, navigate submarines without surfacing, or detect underground resources, all without needing to be constantly adjusted by a human operator. The study confirms that when artificial intelligence is grounded in the reality of how the universe works, it can solve problems that neither pure math nor pure physics could solve alone.

Technical Summary: Physics-Guided Machine Learning for Sim-to-Real Calibration of NV Diamond Magnetometers

Problem Statement
Ensemble nitrogen-vacancy (NV) centers in diamond offer a promising platform for robust vector magnetometry with atomic-scale resolution and sub-nanotesla sensitivity, suitable for field-deployable applications in defense and geophysics. However, their transition from controlled laboratory environments to unshielded, real-world settings is hindered by signal degradation caused by temperature fluctuations, strain, microwave power drifts, and background noise. Conventional calibration methods, such as non-linear least-squares fitting (e.g., Levenberg-Marquardt), struggle with low-contrast, asymmetric, or shot-noise-dominated signals and often fail without precise initial parameters. While alternative strategies like look-up tables or real-time solvers exist, they impose severe computational or storage bottlenecks.

Purely statistical machine learning (ML) approaches, such as standard artificial neural networks (ANNs), offer speed and data compression but face critical limitations in this domain. They require exorbitantly large volumes of training data to avoid underfitting, which is often unavailable in high-precision physical experiments. Furthermore, models trained exclusively on clean synthetic data suffer from a severe "simulation-to-reality" (sim-to-real) gap, failing to generalize to experimental data containing hardware non-idealities. Conversely, training solely on sparse, noisy experimental data leads to underfitting and poor generalization to changing field dynamics.

Methodology
The authors propose a physics-guided hybrid machine learning framework designed to bridge the sim-to-real gap and overcome data scarcity. The core innovation involves embedding the deterministic Zeeman splitting relationship directly into the learning pipeline.

Physics-Guided Architecture: Unlike purely statistical models that treat Optically Detected Magnetic Resonance (ODMR) spectra as high-dimensional input arrays, this framework incorporates the physical law governing the system: the Zeeman splitting relation $\Delta\nu = 2\gamma_{NV}B_z$ . The model uses peak-finding methods to isolate resonance lineshapes and assigns physics-guided labels based on this relationship. This acts as a regularization constraint, restricting the optimization manifold to physically consistent solutions and suppressing spurious local minima.
Hybrid Data Generation: To address the scarcity of high-density experimental data, the authors developed a pipeline that combines scalable synthetic data generation with sparse empirical measurements.
- Synthetic Data: A spin Hamiltonian is solved to generate clean, symmetric ODMR profiles.
- Hybrid Calibration: Sparse, raw experimental ODMR spectra are ingested to lock in baseline physical parameters (e.g., linewidth, asymmetry). These parameters are then used to drive a parameterized simulation engine that generates a dense matrix of synthetic spectra (14,456 spectra) that incorporate real-world non-idealities (e.g., Gaussian shot noise, power broadening) while spanning a wide dynamic range (0–120 G).
Ensemble Learning: The framework employs an ensemble approach where the outputs of a purely stochastic model and the physics-guided model are fed into a final machine learning layer. This multi-channel architecture is designed to screen out remaining statistical fluctuations.
Validation Strategy: The models were evaluated using K-fold cross-validation on synthetic data and tested on completely excluded, uncalibrated raw experimental ODMR spectra to assess sim-to-real performance.

Key Contributions

Embedding Physical Laws: The paper demonstrates that directly embedding the Zeeman splitting equation into the neural network training process significantly enhances predictive accuracy and convergence speed compared to purely statistical baselines.
Hybrid Training Pipeline: A novel workflow is introduced that uses sparse experimental data to initialize a parameterized simulation engine. This allows for the generation of large-scale, realistic training datasets that capture hardware non-idealities without requiring massive physical data collection.
Ensemble Architecture: The integration of stochastic and physics-guided models into an ensemble network further refines predictions, effectively filtering statistical noise.

Results
The study presents quantitative comparisons across three architectures: a purely stochastic model, a physics-guided model, and the final ensemble model.

Precision Improvement: The physics-guided model reduced the average tracking error ( $\Delta$ ) from 0.189 G (stochastic) to 0.00142 G, representing a 99.25% reduction. The final ensemble model achieved an average error of 0.00051 G, a 372-fold precision improvement over the stochastic baseline.
Convergence Stability: While the stochastic model exhibited severe volatility and failed to settle into a global minimum during cross-validation, both the physics-guided and ensemble models achieved immediate, synchronized stability within a few epochs.
Low-Field Performance: The stochastic model suffered catastrophic tracking failure in low magnetic field regimes ( $B < 1$ G), whereas the physics-guided framework smoothed out this divergence.
Experimental Validation: When deployed on uncalibrated, raw experimental ODMR data (completely excluded from training), the hybrid model achieved a mean absolute error (MAE) of 7.97 G and a root mean square error (RMSE) of 9.52 G. The near-zero systematic bias confirmed that the model successfully prevented drift and overfitting to synthetic artifacts, with residual errors bounded by intrinsic hardware fluctuations.

Significance
The authors claim this work establishes a scalable paradigm for self-calibrated sensors, enabling NV magnetometers to operate in unshielded or GPS-denied environments without reliance on external references. By demonstrating that physical laws can be structurally implemented into ML frameworks to bypass data scarcity, the paper posits that this approach is generalizable to other data-scarce physical systems. The framework offers a pathway toward edge-deployable neural networks optimized for diverse metrological applications, moving beyond the limitations of purely statistical training and the "reality gap" inherent in simulation-only approaches.

Technical Summary: Physics-Guided Machine Learning for Sim-to-Real Calibration of NV Diamond Magnetometers

More like this