🤖 machine learning

Segmentation and Classification of Pap Smear Images for Cervical Cancer Detection Using Deep Learning

This study proposes a deep learning framework combining U-Net segmentation and classification for cervical cancer detection, finding that while image segmentation yields marginal improvements in precision and F1-score on the Herlev dataset, its overall impact on classification performance remains limited.

Original authors: Nisreen Albzour, Sarah S. Lam

Published 2026-06-24

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Nisreen Albzour, Sarah S. Lam

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Finding a Needle in a Haystack

Imagine you are a doctor trying to find a few specific, dangerous weeds (cancer cells) hidden inside a giant, messy garden (a Pap smear slide). Right now, doctors have to look at these slides under a microscope one by one. It's slow, tiring, and sometimes they might miss a weed or mistake a harmless flower for a weed because they are tired or the picture is blurry.

This paper asks a simple question: If we use a computer to first "cut out" just the weeds and throw away the dirt and background noise, will the computer get better at spotting the dangerous ones?

The Setup: The Garden and the Tools

The researchers used a famous digital garden called the Herlev Dataset. It contains 917 pictures of cells. They split these into two groups:

Normal: Healthy cells (the good flowers).
Abnormal: Cancerous or pre-cancerous cells (the bad weeds).

They built two "digital detectives" (AI models) to sort these pictures:

Detective A: Looks at the whole messy picture, weeds, dirt, and all.
Detective B: First uses a special tool called U-Net to carefully cut out just the cells, removing the background noise. Then, it looks at the clean, cut-out cells.

The Experiment: Cleaning the Lens

To make sure the test was fair, the researchers didn't just split the data in half. They used a method called 5-fold cross-validation. Think of this like a relay race where the data is split into five teams. The AI trains on four teams and tests on the fifth, then rotates so every team gets a turn being the test group. This ensures the AI isn't just memorizing the answers but actually learning the rules.

They also used Data Augmentation, which is like taking a photo of a flower, then taking more photos of it by flipping it upside down, rotating it, or changing the lighting. This teaches the AI that a weed is still a weed even if it's tilted or in the shade.

The Results: Did Cleaning Help?

The researchers compared how well Detective A (messy pictures) and Detective B (clean, segmented pictures) performed. Here is what they found:

Accuracy (Getting the right answer overall): Both detectives did almost the exact same job. Detective A got it right 81.02% of the time, while Detective B got it right 80.80% of the time. In fact, the messy picture detective was slightly better at the overall score!
Precision (Not crying wolf): Detective B was slightly better at not calling a healthy cell "sick." It improved by 0.41%.
The "Balanced" Score (F1-Score): This is a score that tries to balance being accurate with not missing anything. Here, Detective B (the one that looked at the clean, cut-out cells) did slightly better, improving by 1.30%.

The Conclusion: A Tiny Step, Not a Giant Leap

The main takeaway is surprisingly modest. The researchers found that cutting out the background (segmentation) helped the computer get a slightly more balanced view, improving the "F1-score" by just over 1%.

However, the paper explicitly states that this improvement is very small. The "clean" images didn't make the computer a superhero; it just made it slightly less confused about the difference between healthy and sick cells. The impact of segmentation on the final classification performance appears to be limited.

What's Next?

The authors suggest that while their "cleaning" method worked a little bit, it might not be the magic bullet. They propose that future work should look at more advanced tools (like "Vision Transformers," which are like a different, more powerful type of camera lens) and test these systems even more rigorously to see if they can truly help doctors diagnose cancer earlier and more reliably.

In short: Cleaning up the picture helped the computer a tiny bit, but it didn't solve the whole problem. The computer is still learning how to spot the weeds in the garden.

Technical Summary: Segmentation and Classification of Pap Smear Images for Cervical Cancer Detection Using Deep Learning

Problem Statement
Cervical cancer remains a leading cause of cancer-related mortality among women globally, with early detection via Pap smear tests being critical for reducing death rates. However, the manual examination of Pap smear slides is labor-intensive, prone to human error, and suffers from inter-observer variability. While Deep Learning (DL), specifically Convolutional Neural Networks (CNNs), has shown promise in automating medical image analysis, the specific impact of preprocessing steps—particularly image segmentation—on classification performance in cervical cytology requires further empirical validation. This study addresses the need to determine whether isolating cervical cells from background noise via segmentation yields a measurable improvement in diagnostic accuracy compared to using raw images.

Methodology
The research employed a systematic framework utilizing the Herlev Pap Smear Dataset, a benchmark collection of 917 cervical cell images. The methodology proceeded through the following stages:

Dataset Restructuring: The original seven-class dataset was restructured into a binary classification problem: Normal (healthy epithelial cells, $n=242$ ) and Abnormal (dysplastic and carcinoma cells, $n=675$ ).
Preprocessing and Segmentation:
- Images were resized to $128 \times 128$ pixels and normalized.
- A U-Net architecture was utilized to generate segmentation masks, isolating cervical cells from the background. The masks were post-processed to rescale pixel values, effectively reducing background noise for the classification phase.
Data Augmentation: To address limited data availability and prevent overfitting, techniques including flipping, rotation, and contrast adjustments were applied.
Classification Architecture: A custom CNN was designed for binary classification. The architecture comprised:
- Three convolutional layers with filter sizes of 32, 64, and 128, each followed by max pooling.
- ReLU activation functions.
- L2 regularization and dropout layers to mitigate overfitting.
- A fully connected layer (128 neurons) and a softmax output layer.
- Training was optimized using the Adam optimizer with categorical cross-entropy loss.
Evaluation Protocol: To ensure robust generalization, 5-fold cross-validation was implemented rather than a simple train/test split. Performance was evaluated on two distinct pipelines: one trained on non-segmented (raw) images and another on segmented images. Metrics included accuracy, precision, recall, and F1-score.

Key Contributions

Comparative Analysis: The study provides a direct empirical comparison between CNN performance on raw Pap smear images versus U-Net segmented images, isolating the variable of segmentation to measure its specific contribution to diagnostic accuracy.
Framework Implementation: It establishes a baseline deep learning pipeline integrating U-Net for segmentation and a compact CNN for classification, validated through rigorous cross-validation.
Interpretability Context: The work acknowledges the role of Class Activation Maps (CAMs) and explainable AI in enhancing clinical trust, positioning the proposed framework as a step toward transparent automated diagnostics.

Results
The experimental results indicated that the inclusion of segmentation yielded only marginal improvements in classification performance:

Accuracy: The non-segmented model achieved 81.02%, while the segmented model achieved 80.80% (a decrease of 0.22%).
Precision: Improved slightly from 80.44% (non-segmented) to 80.85% (segmented), a gain of +0.41%.
Recall: Decreased from 81.02% to 80.80%.
F1-Score: Showed the most notable improvement, rising from 78.25% to 79.55%, a gain of +1.30%.

While the segmented model demonstrated a more balanced performance between precision and recall (indicated by the F1-score improvement), the overall classification accuracy remained comparable between the two approaches. The authors noted that the segmented model exhibited less fluctuation in validation accuracy during training, suggesting it may learn more discriminative features, yet the final performance gains were limited.

Significance and Claims
The paper modestly concludes that while segmentation aids in feature extraction and noise reduction, its impact on the final classification performance for cervical cancer detection is limited within the scope of this specific CNN architecture and dataset. The study does not claim that segmentation is a definitive solution for improving diagnostic accuracy but rather suggests it offers a "slightly more balanced classification performance."

The authors position this work as a supplemental tool for clinical applications that aids pathologists, rather than a replacement for manual review. They explicitly frame their findings as a baseline for future research. The paper argues that future work should move beyond simple CNN comparisons to evaluate more complex architectures (such as Vision Transformers), hybrid ensembles, and calibration-aware pipelines. The study emphasizes that future evaluations must prioritize statistical robustness, calibration, and explainability across different cohorts, noting that model complexity does not always guarantee robust generalization without rigorous validation protocols.