A Cross-Modal Detection-Segmentation Framework for Video-Based Process Monitoring in Wire Arc Additive Manufacturing
This study establishes a real-time computer vision pipeline for Wire Arc Additive Manufacturing that identifies YOLOv8n as the optimal detector for weld monitoring and proposes a YOLO-guided SAM2 framework for segmentation, while highlighting the current limitations of foundation models in distinguishing continuous welding features under extreme arc glare.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the world of modern manufacturing, there is a growing desire to build large, complex metal parts not by cutting them from a solid block, but by adding them layer by layer. This process, known as wire arc additive manufacturing, works much like a robotic 3D printer that uses a welding torch. A machine feeds a metal wire into a high-temperature electric arc, melting it and depositing it onto a surface to form a new shape. The success of this entire operation hinges on a tiny, fleeting moment: the molten pool. This is the small, glowing puddle of liquid metal where the wire meets the existing part. If this pool is too hot, too cold, or moves too fast, the final product will be weak or full of cracks. Because this molten metal is hidden under a blindingly bright arc and surrounded by flying sparks, watching it closely is incredibly difficult for human eyes. For years, engineers have relied on sensors that measure electricity or sound, but these methods only offer indirect clues about what is actually happening to the metal.
A new study from researchers at institutions including the Indian Institute of Technology Kharagpur and Robert Gordon University takes a different approach. Instead of listening to the machine or measuring its electrical signals, they decided to teach a computer to see the process directly. They built a system that watches videos of the welding in real time, attempting to identify two critical things: the solid metal that has just been laid down, called the weld bead, and the active, liquid pool of metal right in front of the torch. The challenge is that the video is messy. The bright light of the arc often washes out the image, and the molten metal flows in a way that makes it hard to tell exactly where the liquid ends and the solid begins. The researchers set out to find the best way for a computer to spot these features and then to see if they could use that information to judge the quality of the weld without needing a human to draw detailed outlines on every single frame of video.
To test their ideas, the team gathered a collection of industrial welding videos. They did not use a massive library of millions of images, which is common in other fields of artificial intelligence, but rather a carefully selected set of 335 frames taken from eleven real welding sessions. These frames showed both successful welds and defective ones, capturing a variety of conditions including moments where the arc was unstable or the metal surface was contaminated. The researchers then trained five different types of computer vision models to act as eyes for the machine. They tested a range of approaches, from lightweight models designed to run quickly on standard hardware to more complex systems that use advanced mathematical structures known as transformers. The goal was to see which model could most accurately draw a box around the weld bead and the molten pool, even when the image was distorted by glare or motion.
The results pointed clearly toward one specific type of model. A lightweight system called YOLOv8n proved to be the most effective tool for this job. It successfully identified the weld bead and the molten pool in the vast majority of the test images, achieving a high level of accuracy while processing more than 140 frames every second. This speed is crucial because it means the system could theoretically watch the welding process as it happens and react instantly if something went wrong. In contrast, the more complex transformer-based models, which have shown great promise in other areas of computer vision, struggled significantly. They performed much worse on this small, specialized dataset, suggesting that these powerful systems need far more training data than was available here to learn how to see the subtle details of a welding arc. The study also found that keeping the video in color, rather than converting it to black and white, provided a slight but consistent advantage, indicating that even the faint color differences in the glowing metal help the computer distinguish the parts.
Having found a reliable way to locate the weld, the researchers then tried to take the next step: creating a detailed map of the metal's shape. In many computer vision tasks, once an object is found, a "foundation model" can be used to draw a precise outline around it, separating it pixel by pixel from the background. The team tried using a cutting-edge tool called SAM2, which is designed to understand and segment objects in videos. They used the fast detector to give the tool a rough starting point, and then let the tool track the shape of the weld as it moved through the video. While this worked well for tracking the general area of the weld, it hit a fundamental wall. The tool kept merging the solid weld bead and the liquid molten pool into a single, continuous shape. This happened because, unlike a car or a person which has a clear boundary, the weld bead and the molten pool are physically connected. The metal flows smoothly from liquid to solid, and the intense light of the arc makes the transition blurry. The computer, trained mostly on images of distinct objects like animals or vehicles, could not separate these two continuous parts.
This limitation revealed an important truth about applying general artificial intelligence to specialized industrial problems. While the system could reliably find where the welding was happening and measure the overall size and shape of the deposit, it could not yet perfectly separate the liquid from the solid without human help. The researchers concluded that for now, the best approach is to use the fast detector to find the location and then treat the shape data as an approximation rather than a perfect measurement. They are now planning to use these approximate measurements, combined with the raw video data, to build a system that can judge the quality of the entire weld sequence. By looking at the video as a whole rather than trying to perfect every single frame, they hope to create a system that can tell a factory if a part is good or bad without needing a human to label every single defect. The work establishes a solid foundation for the future, showing that simple, fast computer vision is currently more reliable than complex, data-hungry systems for watching the intense, messy reality of industrial welding.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.