🤖 machine learning

OverNaN: NaN-Aware Oversampling for Imbalanced Learning with Meaningful Missingness

This paper introduces OverNaN, a lightweight oversampling framework that addresses class imbalance in incomplete datasets by generating synthetic samples directly on feature vectors containing missing values, thereby preserving informative missingness patterns rather than erasing them through traditional imputation or deletion.

Original authors: Amanda S Barnard

Published 2026-05-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Amanda S Barnard

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Broken Puzzle"

Imagine you are trying to teach a computer to recognize different types of cars. You show it photos of red sports cars and blue trucks. But, in many of the photos, parts of the image are missing—maybe the license plate is blurred out, or the side mirror is cut off.

In the world of machine learning, these missing pieces are called NaNs (Not a Number).

Traditionally, when data scientists see these missing pieces, they treat them like defects. They have two main ways of fixing them before teaching the computer:

Throwing the photo away: If a photo is missing a mirror, they delete the whole picture. This is bad if you already have very few pictures of that specific car type (a "minority class").
Guessing the missing part: They use math to guess what the missing mirror looks like and fill it in. This is called imputation.

The paper argues that both of these traditional methods are flawed. Throwing data away makes the problem worse if you are already short on data. Guessing the missing parts creates fake certainty—the computer thinks it knows what the mirror looks like, but it's just a guess, which can confuse the model.

The New Idea: OverNaN

The author, Amanda Barnard, introduces a new tool called OverNaN.

Think of OverNaN not as a repair crew, but as a photocopier that respects the missing pieces.

Instead of fixing the missing parts or throwing the photos away, OverNaN says: "Let's keep the missing parts missing, but let's make more copies of these photos so the computer learns better."

This is crucial because in many real-world situations (like science or engineering), missing data isn't an accident; it's meaningful.

The Analogy: Imagine a recipe for a cake. If the recipe says "Add 2 eggs," but you don't have eggs, you don't just guess "2 eggs" and write it down. The fact that the ingredient is missing tells you something about the recipe (maybe it's a vegan version). If you fill it in with a guess, you ruin the recipe. OverNaN keeps the "missing egg" spot blank so the pattern remains true.

How It Works (The "Magic" Trick)

Usually, to teach a computer more about a rare type of car, we use a technique called SMOTE. This works by taking two photos of rare cars and drawing a new, fake photo right in the middle of them.

The Old Way: If one photo is missing a mirror, the old method would guess what the mirror looks like, draw a new car with a "guessed" mirror, and hope for the best.
The OverNaN Way: It looks at the two photos.
- If both have a mirror, it draws a new car with a mirror.
- If one of them is missing a mirror, the new fake car also gets a missing mirror.

It treats the "missingness" as a feature of the data, just like the color or the shape. It creates new synthetic examples that look exactly like the real ones, including their holes and gaps.

Three Ways to Handle the Gaps

The paper explains that OverNaN gives you three different "strategies" for how to handle these missing pieces when making new copies:

The "Conservative" Strategy (Preserve-Pattern): If either of the original photos had a missing piece, the new copy gets a missing piece too. It's very careful not to invent anything new.
The "Fill-in" Strategy (Selective Interpolation): If both original photos have the piece, it fills it in. If only one has it, it leaves the new one blank.
The "Statistical" Strategy (Probabilistic): It looks at how often things go missing in the real world and tries to copy that pattern randomly.

Why Does This Matter? (The Graphene Example)

The paper tests this on a real scientific problem: predicting if a tiny piece of Graphene Oxide (a material used in batteries and medicine) has a specific chemical group attached to it.

The Situation: Some chemical bonds simply don't exist in certain structures. It's not that the scientists forgot to measure them; the bond is physically absent.
The Problem with Old Methods: If you use the old "guessing" method, you might tell the computer that a bond exists when it actually doesn't. This is like telling a chef to add salt to a dish that is supposed to be salt-free.
The OverNaN Result: By keeping the "missing bond" as a missing value, the computer learned to recognize the structure correctly. In the tests, OverNaN was more accurate and consistent than methods that tried to fill in the blanks or delete the data.

The Bottom Line

OverNaN is a tool for when your data is messy, incomplete, and unbalanced (where one type of data is rare).

Instead of trying to "fix" the missing data by guessing or deleting it, OverNaN says: "The missing data is part of the story." It creates more examples of the rare data while keeping the holes in the data exactly where they belong, allowing the computer to learn the truth without being tricked by fake guesses.

It is designed for scientists and engineers working with small, tricky datasets where "what is missing" is just as important as "what is there."

Technical Summary: OverNaN - NaN-Aware Oversampling for Imbalanced Learning with Meaningful Missingness

1. Problem Statement

In applied machine learning, particularly within physical sciences, engineering, and biomedicine, missing values often arise from structural constraints (e.g., destructive measurements, ethical restrictions, or conditional outcomes) rather than stochastic noise. Consequently, the absence of data can carry domain-relevant information. However, standard machine learning pipelines typically treat missing values as defects to be eliminated via listwise deletion or imputation prior to analysis.

This approach is particularly detrimental in imbalanced learning scenarios where minority classes are already under-represented. Deletion disproportionately removes minority samples, exacerbating imbalance, while imputation collapses uncertainty into point estimates, introducing artificial certainty and potential model-dependent bias. Furthermore, widely adopted synthetic oversampling techniques like SMOTE and ADASYN implicitly assume complete feature vectors, forcing practitioners to impute missing data before generating synthetic samples. This process erases the structural missingness patterns before the oversampling occurs, potentially distorting class boundaries and feature space geometry.

2. Methodology: OverNaN

OverNaN is a lightweight, NaN-aware oversampling framework designed to address class imbalance without erasing missingness structure. It extends common synthetic oversampling methods (SMOTE, ADASYN, ROSE) to operate directly on incomplete feature vectors, treating the feature space as $(\mathbb{R} \cup \{\text{NaN}\})^d$ rather than $\mathbb{R}^d$ .

Core Design Principles

Structural Integrity: Missing values are treated as structural elements, not defects. Synthetic samples may legitimately contain NaNs if the original data supports such patterns.
No Artificial Certainty: Oversampling does not introduce inferred values where none were observed, avoiding the encoding of unjustified assumptions.
Explicit Control: Strategies for handling missingness are explicit and user-defined, rather than implicit side-effects of preprocessing.

Algorithmic Implementation

OverNaN modifies the neighborhood definition and sample generation process of existing methods:

Distance Calculation: Distances between samples $x_i$ and $x_j$ are computed only on the set of shared observed dimensions $\Omega_{ij} = \{k : x_{ik} \neq \text{NaN} \land x_{jk} \neq \text{NaN}\}$ .
Feature-wise Generation: Synthetic samples $\tilde{x}$ $\tilde{x}$ are generated feature-wise. For dimensions in $\Omega_{ij}$ $Ω_{ij}$ , standard interpolation occurs ( $\tilde{x}_k = x_{ik} + \lambda(x_{jk} - x_{ik})$ $\tilde{x}_{k} = x_{ik} + λ (x_{j k} - x_{ik})$ ). For dimensions outside $\Omega_{ij}$ $Ω_{ij}$ , the value is determined by one of three explicit strategies:
1. Preserve-Pattern: If a feature is missing in either parent, the synthetic sample retains the NaN.
2. Selective Interpolation: Interpolation occurs only if both parents have observed values; otherwise, the feature remains NaN.
3. Probabilistic Missingness: Missingness is sampled according to empirical per-class missingness rates.

Computational Characteristics

Compatibility: OverNaN operates on NumPy arrays and pandas DataFrames, preserving container types and metadata.
Complexity: It delegates neighbor search to existing libraries. While distance calculations are restricted to observed subspaces, the asymptotic complexity remains similar to the underlying methods ($O(Nd)$ per query in the worst case).
Parallelism: Sample generation is embarrassingly parallel, though OverNaN applies parallelism conservatively at the batch level to avoid race conditions with NaN propagation.
Numerical Stability: It relies on IEEE 754 standard NaN behavior, propagating NaNs deliberately rather than masking them, assuming downstream learners (e.g., tree-based ensembles) are NaN-tolerant.

3. Key Contributions

Framework Development: The introduction of OverNaN, which integrates NaN-awareness into SMOTE, ADASYN, and ROSE without altering their core geometric assumptions, only the domain of definition.
Strategy Definition: The formalization of three distinct strategies for handling missingness during oversampling (Preserve, Selective, Probabilistic), allowing practitioners to align the method with domain knowledge.
Validation Suite: A comprehensive test suite demonstrating functionality across edge cases (e.g., single minority samples, high NaN rates) and integration with downstream classifiers like XGBoost.
Benchmarking: Empirical evaluation on five OpenML datasets with natural missingness and a specific application to a Graphene Oxide Nanoflakes dataset where missingness is chemically meaningful.

4. Experimental Results

Synthetic and Controlled Datasets

Basic Functionality: OverNaN successfully rebalanced classes without imputation. Under the default preserve_pattern strategy, the resampled NaN rate increased (as synthetic samples inherit missingness from parents), whereas ROSENaN maintained rates closer to the original due to its kernel-based perturbation.
Classifier Integration: In a controlled task with XGBoost, all three OverNaN variants improved minority-class recall and F1 scores compared to a baseline. SMOTENaN and ROSENaN achieved perfect recall and F1 (1.000) on the minority class.
Comparative Performance: On a synthetic dataset with 11.9% NaNs, SMOTENaN achieved the highest ROC AUC (0.802) and lowest cross-fold variance compared to pipelines involving feature dropping, instance dropping, or mean imputation followed by standard SMOTE.

Real-World Benchmarks (OpenML)

Evaluated on five datasets (labor, cylinder-bands, Titanic, soybean, car-evaluation):

ROSENaN consistently achieved the highest mean Balanced Accuracy, Macro F1, and ROC AUC across the five datasets.
SMOTENaN provided modest gains over the baseline.
ADASYN-NaN showed improved ROC AUC but slightly lower thresholded metrics on average.
Runtime: ROSENaN remained computationally efficient (0.7s average), while ADASYN-NaN was slower (16.5s) due to adaptive density estimation.

Application: Graphene Oxide Nanoflakes

In a task predicting carboxyl groups on nanoflakes (39.8% NaN rate, where missingness indicates the structural absence of specific bonds):

SMOTENaN outperformed both "Drop Features" and "Imputation + SMOTE" pipelines, achieving a Balanced Accuracy of $0.997 \pm 0.005$ and F1 of $0.999 \pm 0.002$ .
Crucially, SMOTENaN preserved the structural meaning of missing entries (i.e., not inventing bond lengths for non-existent bonds) while maintaining the full feature space, whereas imputation introduced synthetic values and feature dropping discarded informative descriptors.

Imputation vs. OverNaN

On the OpenML anneal dataset (64.98% NaN rate), OverNaN variants produced statistically indistinguishable Balanced Accuracy and Macro F1 scores compared to "Impute-then-Oversample" pipelines. This suggests that imputation is not a strict precondition for competitive performance in these settings.

5. Significance and Claims

The paper posits that missingness is often a signal rather than noise, and eliminating it prior to resampling can distort the learning process. OverNaN's significance lies in:

Preserving Information: It allows practitioners to retain structural missingness patterns that reflect experimental constraints or physical realities, rather than "repairing" data with potentially biased estimates.
Explicit Uncertainty: It encourages the treatment of uncertainty as a first-class consideration, preventing the introduction of artificial certainty into synthetic data.
Domain Applicability: It is specifically targeted at scientific and engineering domains with small, incomplete, and imbalanced datasets where missingness is unavoidable and informative.

The authors explicitly state that OverNaN is not a universal solution for missing data; it does not model missingness mechanisms or infer unobserved values. It is intended as an alternative for settings where downstream learners can natively handle NaNs and where the structural absence of data is meaningful. The framework prioritizes interpretability, control over assumptions, and reproducibility over aggressive optimization or high-throughput capabilities.