🤖 machine learning

NeoJaundice-AI: Smartphone-Based Neonatal Jaundice Detection Using Dual-Input Deep Learning and Synthetic Augmentation

NeoJaundice-AI is an offline, smartphone-based deep learning system designed for India that utilizes dual-input image analysis, synthetic data augmentation, and skin-tone normalization to rapidly and accurately detect neonatal jaundice and estimate bilirubin levels across diverse skin tones.

Original authors: Rahul Patel, Nirjala Jarpula

Published 2026-06-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Rahul Patel, Nirjala Jarpula

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a newborn baby's liver is like a new factory that hasn't fully opened its doors yet. It takes a few days for the factory to start cleaning up a yellow waste product called bilirubin. If the factory is slow, this yellow waste builds up in the baby's blood, turning their skin and the whites of their eyes yellow. This condition is called neonatal jaundice.

While mild cases often fix themselves, severe cases can be dangerous, potentially causing brain damage. The problem is that in rural India, where millions of babies are born, there often aren't enough doctors or expensive machines to test for this. The standard test requires a needle prick and a lab, which many remote clinics don't have.

NeoJaundice-AI is a solution built right into a regular smartphone. Think of it as a "digital doctor's assistant" that lives in your pocket. Here is how it works, broken down into simple parts:

1. The "Two-Eyed" Detective

Most previous apps tried to guess jaundice by looking at just one thing: either the baby's skin or the whites of their eyes.

The Innovation: NeoJaundice-AI is like a detective with two pairs of eyes. It takes two photos: one of the baby's forehead/chest (skin) and one of their eyes (the whites, or sclera).
Why it matters: The paper explains that the yellowing in the eyes often shows up earlier and is easier to see on darker skin tones. By looking at both, the app gets a much clearer picture than apps that only look at one.

2. The "Color Translator" for Darker Skin

A major problem with old medical apps is that they were trained mostly on photos of light-skinned babies. If you showed them a baby with dark skin, the app would get confused, like a translator who only knows English trying to speak Hindi.

The Fix: The researchers built a special "color translator" into the app. Before analyzing the photo, the app adjusts the colors based on the baby's specific skin tone (ranging from light to very dark). This ensures the app measures the yellow of the jaundice, not the brown of the skin.

3. The "Virtual Yellowing" Machine

The app needed to learn what severe jaundice looks like to catch dangerous cases. But in the real world, severe cases are rare, so the app didn't have enough "practice" photos of them.

The Creative Solution: Instead of waiting for more sick babies, the team created a synthetic generator. Imagine taking a photo of a healthy baby and using a digital filter to slowly turn their skin yellow, simulating different levels of sickness.
The Result: They created thousands of "fake" but realistic photos of severe jaundice. This allowed the AI to practice on dangerous cases it might never see in real life, making it much smarter when it actually does.

4. The "Instant Report"

Once the app takes the photos, it doesn't just say "Yes" or "No."

What it does: It acts like a speedometer. It tells you two things:
1. The Severity: Is the baby Normal, Mild, Moderate, or Severe?
2. The Number: It estimates the exact level of yellow waste (bilirubin) in the blood, down to a specific number (like 12.5 mg/dL).
Speed: It does all this in under 3 seconds.

5. The "Offline" Superpower

Most smart apps need the internet to work. But in rural villages, the internet can be spotty or non-existent.

The Advantage: The researchers shrunk the app down to a tiny size (8.3 MB, smaller than a few high-res photos). It runs completely offline on a basic $50 Android phone. No Wi-Fi, no data plan, no lab equipment needed.

The Results

When the team tested this system:

It correctly identified jaundice in 93.5% of cases (meaning it rarely misses a sick baby).
It was accurate even for babies with dark skin tones, where other systems often fail.
It estimated the bilirubin level with an error margin of only 1.4 mg/dL, which is very close to the real lab test.

The Bottom Line

NeoJaundice-AI is a tool designed to bring high-tech medical screening to the places that need it most. It uses a smartphone to take two photos, adjusts for the baby's skin color, uses "virtual" training data to recognize severe cases, and gives an instant, offline report.

Important Note from the Paper: The authors are clear that this app is a screening tool, not a final diagnosis. It's like a smoke detector: if it beeps, you need to call a professional (a doctor) to confirm the fire with a real test. It is designed to flag babies who need help, not to replace the doctor's final decision.

Technical Summary: NeoJaundice-AI

Problem Statement

Neonatal jaundice (hyperbilirubinemia) affects approximately 15–20 million newborns annually in India, with severe cases capable of causing permanent brain damage (kernicterus) or death. While early detection is critical, standard diagnosis relies on invasive serum bilirubin (SB) blood tests requiring laboratory equipment and trained technicians. Non-invasive transcutaneous bilirubinometers (TcB) are prohibitively expensive ( $2,000–$ 8,000 USD) for rural primary health centers (PHCs). Consequently, many cases in rural India go undetected until symptoms are severe.

Existing AI-based solutions face three primary limitations:

Single-modality input: Most systems analyze either skin or sclera (eye white) images, missing the complementary diagnostic value of scleral icterus, which appears earlier and is more visible on dark skin.
Skin-tone bias: Models trained on public datasets dominated by light-skinned infants perform poorly on the darker complexions (Fitzpatrick Types IV–VI) common among Indian newborns.
Data scarcity: Severe jaundice cases are rare in datasets due to their medical urgency, leading to class imbalance and poor model performance exactly where accuracy is most critical.

Methodology

The authors propose NeoJaundice-AI, a smartphone-based screening system designed for offline deployment on commodity Android hardware. The system estimates jaundice severity and predicts serum bilirubin levels in under three seconds without internet connectivity.

1. Model Architecture

The core is a dual-branch EfficientNet-B0 architecture:

Dual-Input Fusion: Two parallel branches independently process a skin image (forehead/chest) and a sclera image.
Feature Fusion: Each branch outputs a 256-dimensional embedding. These are concatenated with a 16-dimensional handcrafted feature vector derived from YCbCr color statistics (mean and standard deviation of Y, Cb, Cr, and ratio features).
Multi-Task Learning: The fused 528-dimensional vector is processed by a shared MLP and split into two heads:
- Classification Head: Predicts four severity classes (Normal, Mild, Moderate, Severe) based on Bhutani nomogram thresholds.
- Regression Head: Estimates continuous bilirubin levels in mg/dL using a Softplus activation to ensure non-negativity.

2. Preprocessing and Skin-Tone Normalization

To address bias against darker skin tones, the system employs a preprocessing pipeline:

White Balance: Removes lighting color casts using the grey-world assumption.
Skin-Tone Classification: Calculates the Individual Typology Angle (ITA) using CIELab components to categorize skin tone.
Adaptive Scaling: Applies a skin-tone-specific affine transform to the Cb and Cr channels, ensuring that a specific bilirubin level maps to a consistent target color regardless of the baseline complexion.

3. Synthetic Data Augmentation

To overcome data scarcity for severe cases and dark skin tones, the authors utilize a synthetic jaundice generator:

Mechanism: Instead of using complex Generative Adversarial Networks (GANs), the system manipulates the YCbCr color space of normal neonatal images. It shifts the Cb (blue-difference) and Cr (red-difference) channels to simulate bilirubin-induced yellowing.
Physics-Informed: The magnitude of the shift is scaled based on the ITA angle to ensure the visual yellowing is perceptually consistent across different skin tones.
Impact: This process increased the "Severe" class samples from 83 to 883 (a 10x increase) and expanded the dataset from 1,847 to 5,047 images.

4. Training and Deployment

Loss Function: A weighted sum of Cross-Entropy (classification) and Mean Squared Error (regression) losses.
Optimization: The model was trained for 50 epochs using Adam optimizer with early stopping.
Mobile Optimization: The model was converted to ONNX format and quantized to INT8, reducing the size to 8.3 MB. This allows for inference on mid-range Android CPUs in approximately 2.1 seconds (p95 latency).

Key Results

Evaluated on a test set of 758 images, NeoJaundice-AI achieved the following performance metrics:

Overall Accuracy: 91.8%
Clinical Sensitivity: 93.5% (correctly identifying jaundiced cases)
Specificity: 95.1%
Bilirubin Estimation: Mean Absolute Error (MAE) of 1.4 mg/dL and RMSE of 1.9 mg/dL.
AUC-ROC: 0.971 (macro average).

Ablation Study Findings:
The inclusion of synthetic augmentation and skin-tone adaptation yielded significant improvements, particularly for underrepresented groups:

Dark Skin Accuracy: Improved from 72.3% (without synthetic data) to 87.6% (with synthetic data).
Severe Class Accuracy: Improved from 65.2% to 83.4%.
Overall Improvement: A +10.6% gain in overall accuracy compared to a baseline without synthetic augmentation.

Significance and Claims

The paper claims NeoJaundice-AI is the first India-focused neonatal jaundice AI system to simultaneously address five critical gaps in existing literature:

Dual-Modality Fusion: Combining skin and sclera inputs for richer feature representation.
Skin-Tone Adaptation: Explicit calibration for Fitzpatrick Types III–VI, addressing the bias found in prior systems (which often dropped to ~71.6% accuracy on dark skin).
Multi-Class Severity Grading: Moving beyond binary classification to a four-class system.
Continuous Regression: Providing a numerical mg/dL estimate rather than just a categorical label.
Offline Mobile Deployment: A fully functional, sub-20 MB model that runs without internet on low-cost hardware.

The authors position the system as a screening tool intended for rural health workers, not a replacement for professional clinical judgment or serum testing. The system includes features for offline operation, age-aware risk assessment (via the Bhutani nomogram), and local language support (Hindi), aiming to bring reliable screening to resource-constrained settings where laboratory access is limited.