⚡ electrical engineering

Beyond Nearest Neighbor Interpolation in Data Augmentation

This paper proposes a modified data augmentation pipeline for convolutional neural networks that eliminates reliance on nearest neighbor interpolation to prevent pixel-level annotation errors and high-frequency detail degradation, instead utilizing a mean-based class filtering mechanism and offline generation to achieve performance gains across medical and X-ray image segmentation datasets.

Original authors: Olivier Rukundo

Published 2026-05-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Olivier Rukundo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to recognize and draw outlines around specific objects in photos, like finding tumors in an X-ray or spotting batteries in a pile of e-waste. To make the robot smart, you show it thousands of pictures. But to make it really smart, you need to show it even more pictures by taking the original ones and twisting, turning, stretching, and rotating them. This is called data augmentation.

The problem, according to this paper, is how we create those new, twisted pictures.

The Old Way: The "Pixel-Perfect" Copycat

Traditionally, when we rotate a picture of an object (like a tumor), the computer has to decide what color to paint the new, empty spaces created by the rotation.

For the photo: Computers usually use a smooth, artistic method (like blending colors) to make the image look natural.
For the outline (the mask): To avoid confusion, computers have historically used a "nearest neighbor" method. Think of this like a pixelated copycat. If a pixel was red, the new pixel is just a copy of the nearest red pixel. It doesn't blend; it just snaps to the nearest neighbor.

The Flaw: The author argues this "snapping" method is like trying to draw a smooth circle using only square Lego bricks. It creates jagged, stair-step edges. These jagged edges confuse the robot because they don't match the smooth curves of the real object. It's like giving the robot a map with jagged, inaccurate borders; it learns to draw jagged borders instead of smooth ones.

The New Way: The "Smooth Painter" with a Safety Net

The author proposes a new approach: Stop using the "snappy" copycat for the outlines. Instead, use the same smooth, artistic blending methods used for the photos, even for the outlines.

The Risk: If you blend colors on an outline, you might get weird, "undefined" colors (like a pixel that is 50% red and 50% blue). The robot doesn't know what to do with a "half-red" tumor.

The Solution: The author invented a Safety Net (called the Global Filter).

Paint Smoothly: First, use the smooth blending method to rotate and stretch the outline. This preserves the fine details and high-frequency structures (the tiny, sharp edges) that the "snappy" method destroys.
The Safety Net: After painting, the Safety Net looks at every pixel. If a pixel is clearly one color or the other, it keeps it. If a pixel is a confusing "half-and-half" mix, the Safety Net checks the average color of the surrounding area and forces it to be either fully one color or fully the other.

The Analogy: Imagine you are stretching a rubber band with a drawing on it.

Old Way: You stretch it, but the drawing gets blocky and pixelated, losing its shape.
New Way: You stretch it smoothly so the drawing stays clear, but then you use a stamp to fix any smudges that appeared during the stretch, ensuring the lines are still sharp and clear.

What Happened When They Tried It?

The author tested this on three different medical image sets (brain scans, breast tissue, and colon polyps) and a battery detection set. They used three different "student" robots (neural networks named U-Net, SegNet, and DeepLabV3+) and one object detector (YOLO).

The Results:

For the "Students" (Segmentation): In most cases, the robot trained with the "Smooth Painter + Safety Net" method performed better. It learned to draw cleaner, more accurate outlines. Specifically, for the U-Net and SegNet models, using the smooth blending method (Bicubic) for the outlines was a clear winner.
For the "Detector" (Object Detection): When looking for batteries, the smooth method also helped the robot find objects more reliably and with higher confidence, though it sometimes found slightly fewer total objects compared to other methods.

The Takeaway

The paper concludes that we shouldn't be afraid to use "smooth" math to handle the outlines of objects, as long as we have a Safety Net to clean up any confusion afterward. By doing this, we stop the robot from learning bad habits (like jagged edges) and help it preserve the tiny, important details of the objects it's trying to recognize.

In short: Don't just copy-paste pixels when twisting your training data. Blend them smoothly, then use a filter to fix the mess. This makes the AI smarter and more accurate.

Technical Summary: Beyond Nearest Neighbor Interpolation in Data Augmentation

Problem Statement
Current deep learning pipelines for semantic segmentation and object detection rely heavily on data augmentation to improve model generalization. However, standard geometric transformations typically employ Nearest-Neighbor Interpolation (NNI or NPI) for categorical labels (masks) to avoid undefined pixel values, while using higher-order methods (e.g., bilinear, bicubic) for image data. The author argues that this reliance on NNI introduces two critical risks:

Exacerbation of Annotation Errors: NNI's reliance on rounding functions creates heavy "jagged edge" artifacts, leading to mismatches between class boundaries and object boundaries, thereby worsening existing pixel-level annotation imperfections.
Loss of Structural Detail: While NNI avoids undefined labels, it ignores the risk of degrading high-frequency structural details. Conversely, standard Extra Pixel Interpolation (EPI) algorithms (bilinear, bicubic) used for images introduce low-pass filtering effects that can degrade structural details in annotated regions.

Existing literature has addressed uncertainty in annotations or label noise but has not sufficiently investigated the specific negative impact of NNI on segmentation model performance or the potential benefits of applying EPI algorithms to categorical labels during augmentation.

Methodology
The author proposes a modified data augmentation pipeline that replaces the standard NNI for masks with EPI algorithms, coupled with a mechanism to handle the resulting undefined categorical labels.

Modified Geometric Transformation: The standard imwarp function is modified to allow the use of EPI algorithms (Bilinear and Bicubic) for transforming categorical masks, rather than being restricted to NNI.
Mean-Based Class Filtering (Global Filter - GF): To mitigate the risk of generating "undesired" pixel values (undefined labels) when using EPI on masks, a filtering mechanism is integrated directly into the augmentation process. For binary segmentation, this Global Filter uses the mean ( $\mu$ $μ$ ) of the transformed mask's pixel values as a threshold:
- If a pixel value $P(x,y) > \mu$ , it is set to the foreground class (255).
- If $P(x,y) \le \mu$ , it is set to the background class (0).
- Original 0 or 255 values are preserved.
- Note: The paper acknowledges this implementation was tested on binary tasks but suggests it can be extended to multiclass scenarios via class-specific thresholding.
Offline Augmentation Pipeline: An offline pipeline was implemented to generate interpolation-specific augmented datasets. This allowed for a controlled quantitative assessment of low-pass filtering effects. The pipeline generated two augmented copies for every original training image.
Datasets and Models:
- Semantic Segmentation: Evaluated on three medical imaging datasets (Breast Histology, MRI Brain, Colon Polyps) using U-Net, SegNet, and DeepLabV3+ (with ResNet50 backbone).
- Object Detection: Evaluated on XBAT+ datasets (Battery-WEEE identification) using YOLOv26.
Evaluation: Performance was measured using Accuracy, Recall, Precision, Intersection over Union (IoU), F1-score (BF1), meanBFScore, Dice score, and mAP@50. Results were analyzed via raw scores and rank-based sums to visualize performance differences.

Key Contributions

Rejection of NNI for Masks: The work challenges the routine use of NNI for mask augmentation, demonstrating that it is not the only solution for undefined labels and often degrades performance by exacerbating annotation errors.
Integrated Filtering Mechanism: The proposal of an internal, mean-based filtering mechanism (GF) that allows the safe use of EPI algorithms on categorical labels without producing undefined classes.
Offline Augmentation Framework: The creation of an offline pipeline to generate and store interpolation-specific training data, enabling a direct comparison of how different interpolation methods (NPI vs. EPI) affect model performance and structural detail preservation.
Comprehensive Empirical Analysis: A systematic evaluation across multiple architectures (U-Net, SegNet, DeepLabV3+, YOLOv26) and diverse datasets (medical and industrial) to determine the optimal interpolation strategy.

Experimental Results
The study yielded mixed but insightful results depending on the architecture and task:

Semantic Segmentation:
- U-Net and SegNet: The BIC_BIC configuration (Bicubic interpolation for both images and masks) generally outperformed the traditional NEA_NEA (Nearest-Neighbor for both) across most metrics (Accuracy, Dice, IoU, meanBFScore), particularly on the Brain and Colon datasets. Qualitative results showed BIC_BIC masks were more compact and closer to ground truth, whereas NEA_NEA introduced false-positive regions.
- DeepLabV3+: The results were less definitive. While rank-based sums sometimes favored NEA_NEA, raw score tables indicated that hybrid configurations (e.g., NEA_BIL or NEA_BIC) often achieved the best performance, suggesting that the optimal strategy is architecture-dependent.
Object Detection (YOLOv26):
- Bicubic (BIC) augmentation provided the highest mAP@50 and Precision for both RGBD and HQD datasets, producing more confident and cleaner detections.
- Bilinear (BIL) augmentation consistently yielded the highest Recall, detecting more objects but with slightly lower precision compared to BIC.
- NEA generally performed lower than EPI methods in terms of mAP and Precision.

Significance and Claims
The paper claims that the choice of interpolation algorithm in geometric transformations is not a neutral preprocessing step but a critical factor influencing model performance.

Performance Gains: The study demonstrates that moving beyond NNI to EPI algorithms, when combined with appropriate filtering, can yield performance gains across multiple quantitative metrics.
Structural Preservation: The proposed framework contributes to the reduction of annotation error exacerbation and the preservation of high-frequency structural details within annotated regions.
Context-Dependent Optimization: The work highlights that the "best" interpolation strategy is not universal; it depends on the specific CNN architecture and dataset characteristics. For instance, while U-Net and SegNet benefited significantly from full EPI (BIC_BIC), DeepLabV3+ showed varied results, and YOLOv26 showed a trade-off between Precision (favored by BIC) and Recall (favored by BIL).
Future Applicability: The author concludes that the proposed framework is extensible to other model architectures and imaging datasets, advocating for flexible interpolation choices in data augmentation pipelines rather than defaulting to nearest-neighbor interpolation.

The Old Way: The "Pixel-Perfect" Copycat

The New Way: The "Smooth Painter" with a Safety Net

What Happened When They Tried It?

The Takeaway

Technical Summary: Beyond Nearest Neighbor Interpolation in Data Augmentation

More like this