⚡ electrical engineering

Vision-Based Crack Segmentation and Coverage Planning for Autonomous Mobile Robots

This paper presents an autonomous mobile robotic system that integrates a lightweight YOLOv11n-seg deep learning model for real-time crack detection with a memory-augmented boustrophedon coverage path planner, achieving high area coverage and path efficiency in simulated infrastructure inspection tasks.

Original authors: Md. Basim Al Zabir Shammo, Tarun Debnath, Lamia Islam, Shohag Sarker, Jakaria Habib, Akifur Rahman

Published 2026-08-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Md. Basim Al Zabir Shammo, Tarun Debnath, Lamia Islam, Shohag Sarker, Jakaria Habib, Akifur Rahman

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Concrete structures like bridges, tunnels, and highways are the silent backbone of modern society, but they are constantly under attack from time and stress. Over years of use, these surfaces develop cracks that, if left unnoticed, can grow into dangerous failures. For decades, the only way to find these hidden fractures has been for human inspectors to climb, crawl, and peer closely at aging infrastructure. This work is slow, expensive, and often puts people in harm's way, especially when they must inspect high walls or tight, unstable spaces. To solve this, engineers are turning to robots that can move on their own, see defects with cameras, and map out a path to check every inch of a surface without human help. The challenge lies in teaching these machines to do two difficult things at once: spot tiny, irregular cracks in a chaotic visual environment, and navigate around obstacles to ensure they do not miss a single spot.

A team of researchers from universities in Bangladesh has built a prototype system that brings these capabilities together. They created an autonomous robot designed to inspect infrastructure by combining a smart camera that can identify cracks with a navigation system that plans a complete path across a surface. The robot uses a camera to look for damage, relying on a type of artificial intelligence trained on thousands of images of real cracks. This system is designed to be lightweight enough to run on small computers, allowing the robot to make decisions quickly while moving. When the robot sees a crack, it marks it; when it sees an obstacle, it stops or turns. The goal is to have the robot sweep back and forth across a wall or road, covering the entire area systematically, much like a person mowing a lawn, but with the added intelligence to avoid furniture, pipes, or other barriers that might block the way.

To test this idea, the researchers did not send a physical robot into a real building immediately. Instead, they built a detailed virtual world inside a computer simulation called Webots. In this digital environment, they placed a small, wheeled robot equipped with a camera and eight infrared sensors that act like touch-sensitive whiskers to detect nearby objects. The team programmed the robot with a specific mission: to travel across a defined area, find cracks, and avoid three large barrel-shaped obstacles placed in its path. The robot's brain consists of two main parts working in tandem. The first part is a vision system trained on a dataset of over 4,000 images of cracks. This system learned to recognize the visual patterns of fractures, distinguishing them from shadows or surface stains. The second part is a navigation planner that breaks the area into a grid and tells the robot to move in a zigzag pattern, ensuring it covers every square inch. If the robot encounters an obstacle, it uses a memory system to remember which spots are blocked and quickly chooses a new nearby target to visit, rather than getting stuck or wasting time.

The results of this simulation were promising but also revealed specific areas where the system needs improvement. The vision system proved highly effective at its primary job. When tested, it correctly identified cracks in about 89 percent of the cases where it was sure of its finding, and it successfully located roughly 70 percent of all the cracks present in the test images. This means the robot is very good at spotting damage when it sees it, though it occasionally misses very faint or tiny cracks. The navigation system also performed well in open spaces, achieving a 94 percent coverage of the total area and moving along its planned path with 92 percent efficiency. The robot completed the entire inspection mission in about 250 seconds, successfully navigating around the obstacles in most of the environment.

However, the experiment also highlighted a significant weakness in how the robot handles complex, cluttered spaces. When the robot entered areas with many obstacles close together, it sometimes fell into a loop, repeatedly turning back and forth between the same few points without making progress. In one specific cluster of obstacles, the robot wasted nearly 90 seconds—more than a third of the total mission time—stuck in this repetitive behavior. This happened because the robot's decision-making logic was too simple; it would pick the nearest unvisited spot without considering if it was trapped, leading to a cycle of futile movements. While the robot could escape these loops eventually, the delay showed that the current method of planning is not yet robust enough for the most difficult real-world scenarios.

Despite these hiccups, the study demonstrates that it is possible to combine advanced visual detection with autonomous movement on a small, efficient platform. The researchers showed that a robot can use a camera to find structural damage and a set of simple sensors to avoid collisions, all while running on a computer that is small enough to fit on a mobile device. The system successfully proved that a modular approach, where the vision and navigation parts work together but can be improved separately, is a viable path forward. The team acknowledges that before this technology can be used on actual bridges or tunnels, the navigation logic must be upgraded to prevent the robot from getting stuck, and the camera system needs to become even more precise at defining the exact edges of a crack. For now, this work provides a solid foundation, showing that the dream of self-driving inspection robots is moving from theory into a tangible, working reality, ready to one day take over the dangerous and tedious task of keeping our infrastructure safe.

Technical Summary: Vision-Based Crack Segmentation and Coverage Planning for Autonomous Mobile Robots

Problem Statement
Structural health monitoring of critical infrastructure (bridges, highways, tunnels) is currently hindered by traditional manual inspection methods, which are labor-intensive, time-consuming, and hazardous for workers. While autonomous mobile robots offer a promising alternative, existing research often treats crack detection and navigation as isolated problems. Vision-based systems frequently rely on static datasets without addressing real-time processing challenges during motion, while coverage planning algorithms often assume perfect perception and static environments. Furthermore, state-of-the-art methods based on Deep Reinforcement Learning (DRL) or Vision Transformers (ViTs) often demand computational resources incompatible with embedded robotic platforms. There is a critical need for a lightweight, modular system that integrates robust real-time crack detection with efficient coverage path planning and obstacle avoidance, operating within the constraints of embedded hardware.

Methodology
The authors propose an integrated autonomous mobile robotic system validated in the Webots simulation environment using an E-puck differential-drive robot. The system architecture consists of three primary modules:

Perception (Crack Detection): The system employs a lightweight YOLOv11n-seg instance segmentation model. The model was trained on a domain-specific dataset of 4,029 crack images (sourced from Roboflow Universe), divided into training (3,717), validation (200), and testing (112) subsets. The architecture utilizes a CSPNet backbone, PANet for feature aggregation, and segmentation layers. The model is designed for real-time inference on embedded platforms, processing 640×640 pixel inputs.
Navigation (Coverage Planning): The robot utilizes a grid-based boustrophedon cellular decomposition strategy. The workspace is discretized into 0.1m cells. A memory-augmented greedy replanning mechanism is implemented to handle dynamic obstructions: waypoints identified as inaccessible are stored in a "blocked set," and the robot selects the nearest unvisited, unblocked waypoint. This approach avoids redundant exploration without the computational overhead of global replanning.
Sensor Fusion and Control: The system fuses data from a forward-facing camera (for mid-range semantic detection) and eight infrared (IR) proximity sensors (for near-field collision avoidance). A rule-based decision fusion layer categorizes threats to enable anticipatory slowing (via camera) and immediate braking (via IR). Localization is achieved through a loosely coupled GPS-IMU fusion, providing pose estimation ( $x, y, \theta$ ) without the computational cost of an Extended Kalman Filter (EKF) in the simulation context. Motion control is managed via PID regulation and geometric tracking to follow the generated waypoints.

Key Contributions
The paper outlines four specific contributions:

Adaptation of YOLOv11n-seg: The authors successfully adapted the nano-scale YOLOv11n-seg model for crack detection, achieving real-time inference on embedded platforms with competitive accuracy for structural defect localization.
Lightweight Coverage Planner: Development of a grid-based boustrophedon planner enhanced with memory-guided greedy waypoint selection. This method maintains a persistent record of blocked cells to prevent redundant exploration while allowing rapid adaptation to new obstacles via nearest-neighbor replanning.
Multi-Modal Sensor Fusion: Implementation of a dual-layer perception framework combining camera-based semantic detection with IR proximity sensing. This enables both strategic trajectory adjustments and reactive collision avoidance without requiring computationally expensive probabilistic fusion.
Comprehensive Experimental Validation: Rigorous testing in Webots using an E-puck robot, providing detailed metrics on coverage efficiency, path efficiency, and detection accuracy, alongside an honest analysis of failure modes such as oscillatory behaviors in dense obstacle fields.

Results
Experimental validation yielded the following performance metrics:

Vision System: The YOLOv11n-seg model achieved 88.8% precision, 70.2% recall, and 80.5% mAP@0.5 for bounding box detection. For segmentation masks, it achieved 80.2% precision, 61.98% recall, and 66.82% mAP@0.5. The model demonstrated stable learning curves over 50 epochs without overfitting. However, performance dropped significantly at stricter localization thresholds (mAP@0.5:0.95 of 23.05% for segmentation), indicating challenges with pixel-perfect boundary delineation.
Navigation System: The system achieved 94.17% area coverage and 92% path efficiency within a 250-second mission duration. The robot successfully navigated a constrained environment with multiple obstacles.
Failure Analysis: The study identified specific limitations. In densely cluttered regions, the reactive navigation strategy exhibited oscillatory behavior, consuming approximately 36% of the total mission time (90 seconds) in repetitive looping at specific waypoint clusters. This was attributed to the purely greedy replanning heuristic lacking global context. Additionally, initial waypoint tracking instabilities were observed during the system's initialization phase.

Significance and Claims
The paper claims that the proposed modular architecture demonstrates the feasibility of integrating vision-based crack detection with autonomous navigation for structural monitoring applications. By prioritizing low latency and computational efficiency, the system offers a foundation for deployment on physical robotic platforms in real-world scenarios. The authors assert that such systems have the potential to reduce inspection costs by an estimated 40-60% and significantly improve worker safety by eliminating human exposure to hazardous environments (e.g., high elevations, confined spaces).

The authors maintain a modest stance regarding the current system's limitations, acknowledging that the purely greedy replanning strategy is insufficient for complex obstacle fields and that the segmentation model requires refinement for precise crack measurement. They position this work as a baseline for future improvements, specifically suggesting the integration of hybrid global-local planning (A* + DWA), post-processing for segmentation refinement, and a transition to vision-based SLAM for GPS-denied environments.

More like this