Multilabel Corrosion Morphology Classification Using Deep Learning with Multimodality Fusion

Corrosion is the slow, silent enemy of the built world. It eats away at bridges, pipelines, and ships, turning strong metal into brittle dust. While some rust spreads evenly across a surface, the most dangerous kind happens in secret. This is localized corrosion, where tiny pits form and grow deep into the metal, often hidden from view until a structure suddenly fails. These pits are not all the same. Some have smooth bottoms, while others develop jagged, cross-hatched textures or even tiny cracks that can lead to catastrophic breaks. For engineers trying to predict how long a piece of metal will last, knowing exactly what these pits look like is vital. If they can tell the difference between a harmless smooth pit and a dangerous cracked one, they can prevent disasters before they happen.

For decades, experts have looked at these pits under powerful microscopes to sort them by hand. But this is a slow, exhausting job, and human eyes can miss subtle details or disagree on what they are seeing. A team of researchers from universities in Texas, Ohio, and Colorado set out to teach a computer to do this work instead. They wanted to build a system that could look at images of corroded metal and automatically identify the different types of damage, even when multiple types appeared in the same picture. The challenge was that the data was messy. Some damage patterns were rare, making it hard for a computer to learn them, and the images often contained confusing mixtures of textures that looked like more than one thing at once.

The researchers gathered hundreds of images taken with a scanning electron microscope, a tool that creates incredibly detailed pictures of metal surfaces. They focused on stainless steel samples that had been exposed to salty water in a controlled environment. In the lab, they created specific conditions to grow different kinds of corrosion. Some samples developed smooth, polished pits, while others formed complex, rough textures or tiny cracks. Crucially, many samples showed a combination of these features, meaning a single image could belong to more than one category. The team also collected the notes written by the scientists who ran the experiments, such as the temperature, humidity, and how long the metal was exposed. They fed all of this information—both the pictures and the written notes—into three different types of artificial intelligence models to see which one could learn best.

The team tested three distinct approaches. One was a standard, well-known image recognition system. Another was a model trained on millions of images and text descriptions from the internet, giving it a broad understanding of how words and pictures relate. The third was a newer, more complex model designed to spot patterns at different scales. They tried feeding the computers just the images, and then tried adding the experimental notes to see if that extra context helped. They also experimented with a specific training trick: telling the computer to pay extra attention to the rare types of damage that appeared only a few times in their data. Without this trick, the computers tended to ignore the rare, dangerous cracks because they were so busy learning the common, smooth pits.

The results showed that the extra information mattered, but only when the computer was taught to care about the rare cases. When the models were trained without special attention to the minority classes, adding the experimental notes did not improve their performance and sometimes even made it worse. However, once the researchers adjusted the training to weight the rare labels more heavily, the models that used both the images and the notes became the most accurate. The model trained on internet data and text proved to be the most reliable overall, especially at spotting the dangerous cracks. It could identify the correct combination of damage types more often than the other systems. The other two models performed well on the smooth textures but struggled slightly more with the complex, cracked patterns.

The study also revealed that these computers are not perfect. In some cases, the images were so ambiguous that even human experts might disagree on what they were seeing. A pit might have a smooth bottom but also a tiny, faint crack that is hard to define. The computers sometimes guessed that a crack was present when it was not, or missed a subtle texture that was there. This suggests that the difficulty lies not just in the machine's ability to learn, but in the inherent fuzziness of the real world, where nature does not always follow neat categories. Despite these challenges, the system successfully learned to distinguish between the different morphologies with a high degree of accuracy, offering a new tool for engineers.

This work establishes a new way to analyze corrosion that combines visual data with the context of how the damage occurred. By showing that class-weighted training is essential for handling rare defects, the researchers provided a blueprint for how to teach machines to recognize the most dangerous forms of decay. While the system was tested on a specific type of steel in a controlled lab, the approach offers a path forward for automating the inspection of critical infrastructure. It does not replace the need for human judgment, but it provides a consistent, tireless second pair of eyes that can spot the subtle signs of failure before they become a crisis.