⚡ electrical engineering

Full-Self Diagnostics (FSD): Physics-Grounded Visual Biomarker Inference from Smartphone Video via Inverse Problems and Operator Learning

This paper introduces Full-Self Diagnostics (FSD), a unified mathematical framework that leverages physics-based inverse problems and operator learning to accurately infer latent physiological states, such as blood glucose levels, from unconstrained smartphone facial videos, achieving clinically relevant performance that scales with the volume of paired training data.

Original authors: Jonathan Thomas, Harsh Thaker

Published 2026-06-19

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Jonathan Thomas, Harsh Thaker

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Your Phone Camera as a "Super-Doctor"

Imagine your smartphone camera isn't just taking a picture of your face; it's actually listening to a complex conversation happening inside your body.

The paper proposes a new way to check your health (like your blood sugar levels) without poking you with a needle. Instead of a medical device, it uses a standard 9-second video of your face taken with a regular phone. The authors call this Full-Self Diagnostics (FSD).

Think of it like this: If you have a fever, you don't just feel hot; your skin might look flushed, your eyes might look tired, your breathing might change, and you might move slower. A trained spouse can often tell you are sick just by looking at you, even before you take your temperature. This paper tries to teach a computer to do exactly that, but with math so precise it can measure specific chemicals in your blood.

How It Works: The Five-Part Engine

The authors built a mathematical "engine" with five parts to make this work:

The Physics Map (The "How"): They start with the laws of physics. Light bounces off your skin, gets absorbed by your blood and tissues, and comes back to the camera. The paper uses complex math (like a map of how light travels through fog) to understand exactly how your blood sugar changes the way light bounces off your face.
The "Many Clues" Theory (The "Why"): This is the most important part. The paper argues that blood sugar doesn't just change how your skin looks; it changes everything. It changes your heart rate, how your pupils react to light, how you blink, and even how you move your face.
- Analogy: Imagine trying to guess the weather. If you only look at the sky (one clue), you might be wrong. But if you look at the sky, feel the wind, smell the air, and watch how the birds fly (many clues), you are almost certainly right. The paper proves mathematically that adding these "behavioral clues" makes the prediction much more accurate.
The Reverse Puzzle (The "Inversion"): Usually, we know the ingredients (blood sugar) and predict the result (skin color). This paper does the opposite: it sees the skin color and tries to solve the puzzle to find the ingredients. They use special math to make sure this puzzle has a stable, unique answer, even if the lighting is weird or the person has dark skin.
The "Smart Learner" (The "Teacher"): The system learns by comparing its guesses against real, needle-based blood tests (ground truth). Every time it guesses, it checks the real answer and corrects itself. The paper proves that as it sees more and more examples, its mistakes get smaller in a predictable way.
The "Universal Translator" (The "Adaptation"): The system is designed to work on any phone, in any light, on any person. It uses math to translate the "accent" of a dark-skinned person or an old phone camera into the same language as the training data, ensuring fairness across different groups.

The "9-Second Video" Secret

Why 9 seconds? The paper explains that this is the perfect amount of time to catch a full "symphony" of body signals:

Heartbeats: It captures enough heartbeats to analyze heart rate variability.
Breathing: It catches the rhythm of your breath.
Micro-movements: It sees tiny, involuntary facial twitches or eye movements that happen when your blood sugar is off.
Pupils: It measures how your eyes react to light.

All of these happen at the same time, creating a rich "multi-channel" signal that is much harder to fake or miss than a single snapshot.

What the Numbers Say (The Results)

The paper presents results from a study involving 59 people and nearly 39,000 video scans. Here is what they found:

The "Self-Test": The lead author (Jonathan Thomas) tested the system on himself over a long period. He had 7,769 data points.
- Accuracy: The system was correct within a standard medical margin of error about 97.6% of the time (meaning the guess was either spot-on or close enough to be safe).
- Safety: Crucially, in the "danger zone" where a wrong guess could be fatal (predicting low sugar when it's high, or vice versa), the system was wrong only 0.27% of the time.
The "Well-Managed" Patient: For a person whose diabetes was well-controlled (blood sugar didn't swing wildly), the system was even better, with an error rate of 17%, which is close to the performance of dedicated medical finger-prick devices.
The "Labile" Patient: For a person with severe, unpredictable diabetes (blood sugar swinging from very low to very high), the system still worked, proving it can handle extreme cases.
The Learning Curve: The paper shows a graph proving that as they added more data, the system got better exactly as their math predicted. It's like a student getting better at a test the more practice questions they do.

The "Doctor Analogy" Made Real

The paper uses a specific analogy: A trained spouse can often tell their partner is having a low-blood-sugar episode just by looking at them, before any medical device is used. This paper claims to have turned that "spouse intuition" into a rigorous, mathematical formula. It proves that the visual signal contains enough hidden information to recover your health stats, provided you use all the clues (heart, eyes, skin, movement) together.

Important Limits (What the Paper Admits)

The authors are very honest about what they haven't done yet:

It Needs a Teacher: Right now, the system needs to be "supervised." It learns by comparing its video guesses to real blood tests from a CGM (Continuous Glucose Monitor) or a needle. It cannot currently learn without these real blood tests to start with.
It's Still Improving: While the results are promising, the overall accuracy (MARD) is still higher than the best medical devices on the market (which are around 8-10% error). The system is at about 30% error overall, but the authors argue this will drop as they feed it more data.
Extreme Cases: The system is slightly less accurate when blood sugar is extremely high or extremely low, simply because those are rare and hard to learn. However, the math predicts that adding more data from these extreme cases will fix this quickly.

Summary

In short, this paper claims that a 9-second video of your face contains a hidden "fingerprint" of your blood sugar and other health markers. By using physics, advanced math, and a learning system that looks at heartbeats, eye movements, and skin color all at once, they can decode this fingerprint.

They have proven that this method is mathematically sound, safe (very few dangerous errors), and gets better the more data you give it. They are not claiming it is a finished medical product yet, but rather a powerful new framework that turns your phone camera into a potential universal health sensor.

Based on the provided manuscript, here is a detailed technical summary of the Full-Self Diagnostics (FSD) framework.

Problem Statement

The paper addresses the limitations of current non-invasive biomarker sensing, which relies on dedicated, single-purpose hardware (e.g., spectrometers, PPG patches) that is expensive, fragile, and informationally impoverished. The authors propose a paradigm shift: utilizing the ubiquitous consumer-grade smartphone camera as a universal physiological interface. The core challenge is to mathematically prove that a 9-second facial video contains sufficient, invertible information to recover latent physiological states (such as blood glucose) without invasive measurement, despite the camera's limited spectral range (RGB) and the complexity of biological tissue optics.

Methodology: A Five-Component Mathematical Framework

The FSD framework is grounded in five mutually reinforcing theoretical components:

Physics-Grounded Forward Model:
- The system models light propagation through dermal tissue using the Radiative Transfer Equation (RTE) and its Diffusion Approximation.
- It links camera observables to biomarker concentrations ( $x$ ) via Chromophore Decomposition (Beer-Lambert law), mapping absorption coefficients to specific biomarkers (glucose, hemoglobin, melanin, etc.).
- Crucially, it acknowledges that while glucose absorption peaks are in the Near-Infrared (NIR) range (outside standard RGB), glucose modulates light scattering and vascular tone, creating observable effects in RGB channels.
Temporal and Behavioral Correlate Model:
- The framework posits the Multi-Correlate Hypothesis: a biomarker like glucose orchestrates a cascade of downstream effects beyond direct optical absorption.
- The 9-second video is decomposed into five independent channels:
  - Spectral: Static chromophore mixture.
  - Pulse/rPPG: Heart rate and Heart Rate Variability (HRV) modulated by autonomic tone.
  - Respiratory: Breathing patterns (e.g., Kussmaul breathing in ketoacidosis).
  - Micro-expression: Involuntary facial muscle activations disrupted by glucose dysregulation.
  - Oculomotor: Pupillary light reflex (PLR) and saccadic eye movements.
- These channels are treated as conditionally independent evidence sources that strictly increase mutual information.
Regularized Inverse Problem:
- Inferring physiological state from visual data is formulated as a Tikhonov regularized inverse problem.
- The solution is proven to be Lipschitz continuous and stable, ensuring that small perturbations in the visual signal do not lead to catastrophic errors in biomarker estimation.
Operator Learning Formulation:
- Instead of learning a fixed vector-to-vector mapping, the framework models the inference as an operator between function spaces (mapping the full spatio-temporal video function to the physiological state function).
- This allows for resolution invariance (generalizing across 720p/1080p/4K) and device generalization (different camera sensors) without retraining, provided spectral calibration is maintained.
Supervised Learning as Bayesian Inference:
- The learning process is defined as stochastic variational inference, where the model updates its parameters using paired ground-truth biosensor data (e.g., Dexcom G7 CGM).
- The framework proves an $O(1/\sqrt{N})$ convergence rate, demonstrating that error decreases predictably as the dataset size ( $N$ ) increases.
- Domain Adaptation: Uses Wasserstein Optimal Transport to align distributions across different demographics (skin tones, ages) and lighting conditions, minimizing systematic bias.

Key Theoretical Contributions

Information-Theoretic Observability: The paper proves that the visual observation $Y$ has strictly positive mutual information with the physiological state $X$ . Furthermore, it demonstrates that adding temporal/behavioral channels strictly increases the total Fisher Information, lowering the Cramér–Rao lower bound on estimation variance.
Domain-Uniform Identifiability: The framework establishes that the inverse problem is identifiable uniformly across demographic variations, provided the forward map is injective and Lipschitz with respect to demographic variables.
Manifold Hypothesis & Stable Recovery: It posits that physiological states lie on a low-dimensional manifold. Under bi-Lipschitz conditions on this manifold, the inverse operator is stable, ensuring bounded reconstruction error.
Conformal Uncertainty Quantification: The system provides distribution-free, exact coverage guarantees for prediction intervals (Algomash Confidence Bands), a critical requirement for clinical safety.

Empirical Results (as of June 5, 2026)

The paper reports preliminary longitudinal results from 38,812 paired scans across a 59-user cohort.

Lead Author Self-Evaluation (Jonathan Thomas):
- Dataset: 7,769 prospective datapoints (glucose range 35–550 mg/dL).
- Performance: Mean Absolute Relative Difference (MARD) of 29.86%.
- Clinical Safety: 97.57% of predictions fell into Clarke Error Grid Zones A+B (clinically acceptable), with only 0.27% in the dangerous Zone E.
- Convergence: The error reduction from November 2025 to March 2026 matched the theoretical $O(1/\sqrt{N})$ prediction within measurement noise.
Well-Managed Diabetic Participant:
- Achieved a MARD of 17% on a narrower glucose range (70–180 mg/dL), approaching the performance of dedicated fingertip PPG devices in constrained bands.
Labile Type 1 Diabetic Cohort Member:
- Contributed 25,995 scans spanning the extreme range (35–550 mg/dL).
- Achieved a MARD of 46.09%, validating that the multi-channel signal remains informative even in extreme, high-variance regimes, and confirming the scaling law.

Significance and Claims

The paper claims to provide the first mathematically rigorous infrastructure for non-invasive, contactless physiological monitoring using standard smartphone cameras.

Validation of the "Doctor Analogy": It formalizes the clinical observation that trained observers can detect physiological decompensation (e.g., hypoglycemia) via visual cues, proving mathematically that these cues constitute a rich, multi-channel information source.
Scalability and Equity: By leveraging the "zero marginal hardware cost" of existing smartphones, the framework offers a pathway to scalable screening and chronic disease management, particularly in low-resource settings.
Regulatory Pathway: The inclusion of conformal prediction bands and the prospective longitudinal multi-modal panel design are presented as the necessary steps for clinical translation and regulatory approval.

Limitations and Scope

The authors explicitly state current limitations:

Supervised Nature: The system currently requires paired biosensor ground truth for training and updates; it is not yet fully self-supervised.
Data Composition: Quantitative metrics are heavily influenced by self-collected data from the lead author and a few high-volume participants. Broader, independent multi-center validation is ongoing.
Extreme Regimes: Performance (MARD) is lower in extreme glycemic tails (<60 or >350 mg/dL) compared to FDA-cleared CGMs, though the framework predicts rapid improvement with continued data scaling in these regions.
Environmental Factors: Motion artifacts, lighting, and accessories can degrade predictions, necessitating the use of conformal uncertainty bands to flag low-confidence cases.

The paper concludes that the FSD framework successfully demonstrates that consumer-grade video, when combined with physics-grounded priors and operator learning, can recover clinically relevant biomarker panels with bounded, data-scalable error.