💻 computer science

Balancing Real and Synthetic Data for CNN-based Masonry Crack Detection

This paper demonstrates that combining synthetic masonry crack images with a modest amount of real data (specifically an 80% synthetic and 20% real ratio) significantly enhances the performance of CNN-based crack detection models, achieving higher F1-scores and mIoU than training on real data alone.

Original authors: Mattia Forlesi, Alfonso Esposito, Ivan Zyrianoff, Alessandro Marzani, Marco Di Felice

Published 2026-06-09

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Mattia Forlesi, Alfonso Esposito, Ivan Zyrianoff, Alessandro Marzani, Marco Di Felice

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to spot cracks in old brick walls. This is a crucial job because cracks are like the "fevers" of a building; spotting them early prevents the building from getting sick or, worse, collapsing.

Usually, to teach a robot (specifically a type of AI called a Convolutional Neural Network, or CNN), you need to show it thousands of photos of real cracks. But here's the problem: finding and photographing enough real cracks is like trying to find a specific needle in a haystack. It takes a long time, costs money, and the photos you find online might not be good enough for the robot to learn properly.

The Solution: A "Photo Shop" for Cracks
The researchers in this paper came up with a clever workaround. Instead of just hunting for real cracks, they built a digital tool that acts like a "sticker maker." They took photos of normal, healthy brick walls and digitally "pasted" realistic-looking cracks onto them. This created a huge library of synthetic (fake but realistic) crack images.

The Experiment: Mixing Real and Fake
The team wanted to see if they could train the robot using mostly these "sticker" cracks and only a few real ones. They set up a cooking competition with six different recipes (training scenarios):

Recipe A: 100% Real cracks (The traditional way).
Recipe B: 100% Synthetic cracks.
Recipe C through F: Various mixes of Real and Synthetic (e.g., 50/50, 20/80).

They used a specific AI model named InceptionV4 (think of it as the robot's "brain") to taste-test these recipes.

The Surprising Results
Here is what they found, which turned out to be quite counter-intuitive:

The "All Fake" Disaster: When they tried to teach the robot only with synthetic cracks, it failed miserably. It was like teaching someone to drive only on a video game; when they got on a real road, they didn't know what to do. The robot couldn't tell the difference between the fake cracks and the real wall textures.
The "Sweet Spot": The magic happened when they mixed the data. The best result came from a recipe of 80% Synthetic cracks and only 20% Real cracks.
- This mix actually performed better than training with 100% real data.
- It achieved a score of 76% (F1-score) and 80% (Mean IoU), beating the "100% Real" team which scored around 74%.

Why This Matters
Think of it like learning to paint. If you only look at masterpieces (real data), you learn a lot, but it's expensive and hard to get. If you only practice on a computer screen (synthetic data), you learn the theory but lack the feel. The paper shows that if you spend most of your time practicing on the computer (synthetic) and just a little bit of time looking at the real masterpieces (20% real), you end up being a better painter than if you only looked at the real ones.

The Bottom Line
The paper concludes that you don't need to spend years collecting thousands of real photos to build a great crack-detection system. By using a simple tool to generate fake cracks and mixing in a small amount of real data, you can build a system that is just as good, or even better, at finding cracks in masonry walls. This saves time and effort while keeping the building safe.

Technical Summary: Balancing Real and Synthetic Data for CNN-based Masonry Crack Detection

Problem Statement

The identification of cracks in masonry structures is a critical component of Structural Health Monitoring (SHM), serving as an early indicator of building health to prevent damage and accidents. While Deep Learning (DL), specifically Convolutional Neural Networks (CNNs) and semantic segmentation, offers scalable solutions for automated crack detection, these models rely heavily on large, diverse datasets. Collecting sufficient real-world data for complex surfaces like masonry is time-consuming and labor-intensive. Furthermore, publicly available datasets often lack the specificity required for rigorous study. Existing synthetic data generation methods present trade-offs: Generative Adversarial Networks (GANs) can produce high-fidelity textures but often lack domain-expert supervision, leading to unrealistic crack representations relative to background structures, while 3D modeling approaches require significant expertise and may lack aesthetic refinement.

Methodology

The authors propose a pipeline to address data scarcity by generating synthetic crack images to complement a limited real dataset. The methodology consists of four stages:

Data Acquisition:
- Real Data: A custom dataset of masonry crack images was collected from buildings in Bologna and surrounding areas using standard smartphones (Oppo Reno 6 Pro, iPhone 12 & 13) under daylight conditions.
- Synthetic Data: A custom Python-based tool with a Tkinter GUI was developed to overlay semi-transparent crack masks onto high-resolution masonry background images. This tool allows for controlled spatial placement, arbitrary rotation, and scale consistency, preserving the original resolution for export.
Data Preprocessing:
- Raw images were cropped into $224 \times 224$ patches to standardize resolution.
- Ground-truth masks were created via pixel-wise annotation.
- Data augmentation (flips and $90^\circ$ rotations) was applied to expand the training set.
- Images were normalized using ImageNet statistics to ensure compatibility with pre-trained backbones.
Model Architecture:
- The study employed the U-Net encoder-decoder architecture with skip connections.
- Seven different backbone architectures were evaluated: Custom U-Net, VGG16, MobileNetV2, ResNet50, DenseNet169, InceptionV4, and DeepLabV3.
- Most backbones utilized pre-trained weights.
Experimental Setup:
- The real dataset was split into training (60%), validation (20%), and test (20%) sets.
- Synthetic data was introduced only into the training set of the best-performing model.
- Three loss functions were tested: F1-Loss, Focal Loss, and a combination of Cross-Entropy and Dice Loss (CED).
- Training was capped at 100 epochs with early stopping.

Key Contributions

Custom Dataset Creation: The team collected and labeled a specific dataset of masonry crack images, chosen for the material's complexity to rigorously test DL efficiency.
Synthetic Generation Tool: The authors developed a simplified, custom image processing method that overlays crack masks onto masonry textures. Unlike complex GANs or 3D rendering, this approach allows for controlled insertion of cracks with specific parameters.
Evaluation of Data Ratios: The study proposes an evaluation framework to determine the minimum amount of real data required when augmented with synthetic data to achieve or exceed the performance of a real-data-only baseline.

Results

The experiments were conducted in two phases:

Model Selection: Using only real data, various U-Net architectures were compared. InceptionV4 emerged as the best-performing model, achieving a 74.7% F1-score and 78.69% mean IoU (using F1-Loss). This model served as the baseline for subsequent experiments.
Synthetic vs. Real Data Balance: The InceptionV4 model was retrained with varying ratios of synthetic to real data (from 100% synthetic to 100% real).
- 100% Synthetic: Performed poorly (31.6% F1, 57.68% mIoU) due to the domain gap between synthetic training and real testing.
- 20/80 Scenario (20% Synthetic / 80% Real): This configuration yielded the highest performance, achieving a 76.82% F1-score and 80.36% mean IoU.
- 80/20 Scenario (80% Synthetic / 20% Real): This scenario achieved results comparable to the real-data-only baseline, with a 73.48% F1-score and 78.17% mean IoU.

The results indicate that training on synthetic data combined with a modest addition of real data (20%) can outperform training on real data alone. Conversely, a dataset composed of 80% synthetic and 20% real data can match the performance of a 100% real dataset.

Significance and Claims

The paper claims that the proposed method demonstrates the potential of synthetic data to significantly reduce the effort required for data collection and labeling while enhancing crack detection accuracy. By utilizing a simple overlay tool rather than complex generative models, the authors show that a balanced combination of real and synthetic data can improve segmentation metrics (F1-score and mIoU) compared to using real data exclusively. Specifically, the study highlights that the 20/80 scenario (synthetic/real) not only matches but slightly exceeds the performance of the real-only baseline, suggesting that synthetic data can effectively augment limited real-world datasets for masonry crack detection tasks.