💻 computer science

Pixel-Level Pavement Distress Assessment Using Instance Segmentation

This paper presents a Mask R-CNN-based instance segmentation system for precise, pixel-level pavement distress assessment that outperforms YOLO detection in localization accuracy and aggregate crack-area estimation on a custom field-collected dataset, while highlighting key challenges in annotation consistency and class imbalance.

Original authors: Logan Dewick (University of Wisconsin - Green Bay), Bibesh Pyakurel (University of Wisconsin - Green Bay), Kong Pheng Yang (University of Wisconsin - Green Bay), Nazim Choudhury (University of Wiscons

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Logan Dewick (University of Wisconsin - Green Bay), Bibesh Pyakurel (University of Wisconsin - Green Bay), Kong Pheng Yang (University of Wisconsin - Green Bay), Nazim Choudhury (University of Wisconsin - Green Bay), M. G. Sarwar Murshed (University of Wisconsin - Green Bay)

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a road inspector. Your job is to find cracks, holes, and damage on the pavement so the city can fix them before they turn into giant potholes. In the past, you had to walk the streets, squinting at the asphalt, which was slow, tiring, and dangerous near traffic.

This paper is about building a digital inspector that uses a camera and a smart computer brain to do this job automatically. Here is the story of how they built it and what they found.

The Problem: Why "Boxing" Isn't Enough

The researchers explain that earlier computer programs tried to find road damage in two ways:

The "Yes/No" Check: Looking at a small square of road and saying, "Is there a crack here?" (Good for knowing if there is a problem, but bad for knowing where exactly).
The "Box" Method: Drawing a square box around a crack. This is like trying to wrap a long, winding snake in a square gift box. The box covers the snake, but it also covers a lot of empty space, and it doesn't tell you the snake's exact shape or how long it really is.

For road maintenance, you need to know the exact shape and total area of the damage to calculate repair costs. A square box is too clumsy for thin, branching cracks. You need something that can trace the crack like a highlighter pen, pixel by pixel.

The Solution: The "Digital Highlighter" (Instance Segmentation)

The team built a system called Mask R-CNN. Think of this as a super-smart digital highlighter. Instead of just drawing a box, it can:

Find the crack.
Draw a perfect outline around it, following every twist and turn.
Count exactly how many pixels of the road are damaged.
Distinguish between a real crack and things that look like cracks, like oil stains, shadows, or painted road lines.

They tested this on a new dataset they created called UWGB-STREETCRACK. They drove a car with an iPhone 15 Pro Max mounted on the front, recording video of local roads. They then manually drew precise polygon outlines around four types of damage:

Longitudinal cracks: Running parallel to the road.
Transverse cracks: Running across the road.
Alligator cracks: A web of interconnected cracks (like a lizard's skin).
Potholes: Deep holes in the road.

The Experiment: Who Won the Race?

The researchers pitted their "Digital Highlighter" (Mask R-CNN) against two other types of computer vision models:

The "Box Drawer" (YOLO): A fast detector that just draws squares.
The "Crop-Cutter" (DeepSegmentor): A model that only works if you manually cut out the crack from the picture first.

The Results:

The Winner: The Mask R-CNN (specifically the version with a "ResNet-101" brain) was the clear champion.
- It was incredibly accurate at finding cracks (90% recall) and very good at not making mistakes (84% precision).
- The "Area" Test: This is the most impressive part. The actual damaged area in the test photos was 2.170%. The Mask R-CNN estimated it to be 2.164%. That is a difference of less than one-hundredth of a percent! It was almost perfect.
The Loser: The "Box Drawer" (YOLO) struggled badly, with very low scores. It couldn't handle the messy, real-world road conditions well.
The "Crop-Cutter": While it could estimate the area of a crack if you gave it a clean, cropped picture, it failed when looking at a whole road scene because it got confused by shadows and road markings.

The Challenges: Where It Still Stumbles

Even though the system is great, the authors admit it's not perfect yet. They found a few "glitches" in the real world:

Confusing Visuals: Sometimes the computer mistakes a long oil stain or a painted line for a crack.
The "Manhole" Mystery: Areas around manhole covers are tricky. The computer sometimes can't tell if a line near the cover is a crack or just part of the metal cover.
The "Alligator" Puzzle: When there is a huge web of cracks, humans might argue: "Is this one giant crack or five small ones?" The computer sometimes guesses differently than the human annotator. This shows that the rules for drawing the lines need to be very strict.

The Bottom Line

This paper proves that Instance Segmentation (the "Digital Highlighter") is the best tool we currently have for automatically measuring road damage. It doesn't just find the cracks; it measures them with a level of precision that matches human experts almost perfectly.

The authors conclude that while the system is ready to be a practical tool for estimating how much road needs fixing, we still need to work on making the "rules" for labeling the data clearer so the computer doesn't get confused by shadows or stains.

Technical Summary: Pixel-Level Pavement Distress Assessment Using Instance Segmentation

Problem Statement
Automated pavement distress assessment requires geometric precision that exceeds the capabilities of image-level classification or coarse bounding-box detection. While bounding boxes are efficient for deployment, they fail to accurately represent thin, branching, irregular, or disconnected cracks, which are critical for maintenance-relevant quantification such as crack-area estimation and morphological analysis. Furthermore, field-collected roadway imagery presents significant challenges, including visual confounders (shadows, stains, road markings, manhole edges) and the need to distinguish between multiple simultaneous distress instances. Existing literature often relies on cropped regions or single-crack scenes, leaving a gap in evaluating segmentation performance on full-scene, real-world roadway data with multiple distress categories.

Methodology
The authors propose a vision-based system utilizing Mask R-CNN instance segmentation implemented via the Detectron2 framework. The system is evaluated on UWGB-STREETCRACK, a custom dataset acquired using a vehicle-mounted iPhone 15 Pro Max. The dataset comprises 1,874 images (1,643 for training/validation, 231 for testing) annotated with polygon masks for four classes: longitudinal cracks, transverse cracks, alligator cracks, and potholes.

The technical pipeline involves:

Data Preparation: Polygon annotations from Label Studio are rasterized into binary instance masks. Images are resized (short side 800px, long side max 1333px) with aspect ratio preservation.
Model Architecture: Five Mask R-CNN backbone variants were considered, including ResNet-50, ResNet-101 (with FPN and C4 features), and ResNeXt-101. The models were initialized with COCO-pretrained weights and fine-tuned using stochastic gradient descent (momentum 0.9, weight decay 0.0001) over 40 epochs.
Inference and Evaluation: The system performs joint localization, classification, and pixel-level segmentation on full roadway images. Evaluation utilizes a project-specific protocol where predicted instances are matched to ground truth via bounding-box Intersection over Union (IoU $\ge$ 0.1). Metrics include Precision, Recall, F1 score, and aggregate crack-area fraction.
Comparative Baselines: To contextualize the results, the authors adapted a CSPDarknet53-based YOLO detector (trained from scratch on the dataset) and a DeepSegmentor model (adapted as a crop-level reference) to serve as detection and segmentation benchmarks, respectively.

Key Contributions
The paper makes the following specific contributions:

Dataset Documentation: The release and documentation of UWGB-STREETCRACK, a smartphone-based dataset with polygon-level annotations for four distinct pavement distress classes.
Pipeline Implementation: A description of a full Mask R-CNN instance-segmentation pipeline tailored for pavement distress, capable of handling full-scene imagery and estimating aggregate crack area.
Empirical Results: The reporting of matched test results for archived Mask R-CNN experiments, preserving original precision, recall, F1, and area-fraction values without alteration.
Baseline Adaptation: The adaptation of a Mandal CSPDarknet53-based YOLO detector and a DeepSegmentor model on the same dataset to provide object-detection and crop-level segmentation references.
Error Analysis: An identification of methodological gaps, including annotation ambiguity (e.g., distinguishing alligator cracks from multiple linear cracks), minority-class sparsity, and the lack of standard mask-level AP reporting in current project records.

Results
The Mask R-CNN with a ResNet-101 FPN backbone achieved the best performance among the evaluated variants:

Precision: 84.23%
Recall: 90.04%
F1 Score: 87.04%
Area Estimation: The model predicted a crack-area fraction of 2.164%, closely matching the ground-truth fraction of 2.170% (an absolute difference of 0.006 percentage points).

In comparison, the adapted YOLO (CSPDarknet53) detector achieved significantly lower metrics (27.5% precision, 20.7% recall, 23.62% F1) under the validation protocol. The DeepSegmentor model, while producing a similar area estimate (2.130%), was limited to a crop-level evaluation and could not perform full-scene localization or distinguish cracks from visual confounders like shadows and road markings.

Significance and Claims
The paper positions instance segmentation as a practical direction for field pavement imagery and aggregate crack-area estimation. The authors claim that the proposed framework successfully addresses the geometric limitations of bounding-box detectors by providing pixel-level masks that enable precise area calculation.

However, the paper maintains a modest tone regarding its claims:

It explicitly states it is an applied empirical study rather than a proposal of a new network architecture.
It acknowledges that the results are based on project-specific matching protocols and should not be interpreted as standard COCO-style mask AP benchmarks.
It highlights that annotation consistency remains a challenge, noting that ambiguity in labeling (e.g., whether a connected network is one alligator crack or multiple linear cracks) affects both training targets and evaluation counts.
The authors conclude that while the system is effective for the reported tasks, open challenges regarding confounder rejection and standardized mask-level benchmarking must be addressed before the system can serve as a standardized benchmark.

The Problem: Why "Boxing" Isn't Enough

The Solution: The "Digital Highlighter" (Instance Segmentation)

The Experiment: Who Won the Race?

The Challenges: Where It Still Stumbles

The Bottom Line

More like this