Foundation Models for Cold-Start Industrial Visual Anomaly Detection: A Survey
This survey reviews foundation model-based industrial visual anomaly detection under a cold-start deployment framework, systematically categorizing methods by data availability and pipeline roles while addressing challenges in adaptation, evaluation, and scalable implementation.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the vast, humming world of modern manufacturing, the eyes of the factory floor are increasingly digital. For decades, automated quality control has relied on cameras to spot the tiny flaws that human inspectors might miss: a scratch on a car door, a missing screw in an engine block, or a hairline crack in a circuit board. The challenge has always been that these defects are rare and unpredictable. A machine learning system trained to recognize a specific flaw on one product line often fails when the product changes, the lighting shifts, or a new supplier provides slightly different materials. To fix this, engineers traditionally needed to gather thousands of examples of both perfect and broken items to teach the computer what to look for. But in the real world, especially when a new production line starts up, there are often no broken examples to study, and sometimes not even enough perfect ones to build a reliable baseline. This is the "cold start" problem: how do you teach a system to find a needle in a haystack when you have never seen a needle before, and you only have a few straws to show it what a haystack looks like?
A new survey by researchers Xu Tan, Haidong Gao, Ronghua Liang, and Yi Yang explores how a new generation of artificial intelligence, known as foundation models, is beginning to solve this puzzle. These models are massive systems trained on billions of images and text from the internet, giving them a broad, general understanding of the world. They know what a screw looks like, how light reflects off metal, and what a "scratch" means in language. However, they were not built to spot industrial defects; they were built to recognize objects and answer questions. The researchers found that while these powerful tools offer a promising shortcut for cold-start inspection, they are not a magic switch. The survey reveals that simply plugging these models into a factory line does not work. Instead, success depends on carefully separating what the factory has available (like a few photos of perfect products) from how the model is tweaked (using outside data to teach it what a defect looks like). The authors argue that the field has been confused by mixing these different scenarios together, making it hard to tell which methods are truly effective. By sorting the approaches based on exactly what data is available and how the model is adapted, the survey provides a clear map for the future of automated quality control.
The researchers began by defining the specific conditions under which these systems must operate. In a typical factory, a new product line might arrive with no history of defects, or perhaps just a handful of photos showing what a perfect item looks like. In the most difficult scenario, known as the zero-shot regime, the system has no reference images from the target line at all; it must rely entirely on its pre-existing knowledge. In slightly easier scenarios, the system might have a few photos of normal items to compare against, or even a rare photo of a broken item. The survey highlights a critical distinction that previous studies often blurred: the difference between having target data (photos from the specific factory line) and using auxiliary data (photos from other factories or synthetic images used to train the model). The authors insist that these two factors must be reported separately. A method that performs well because it was secretly trained on thousands of external defect images is fundamentally different from one that works with nothing but a few photos of the current product. Without this clarity, it is impossible to know if a system is truly smart or just overfitted to a specific dataset.
The survey then categorizes how these foundation models are actually being used to find defects, breaking the process down into three main stages. The first stage is preparation, where the system must decide what part of the image to look at. Industrial images are often cluttered with backgrounds, shadows, and complex machinery. To solve this, researchers are using large segmentation models to isolate the object of interest, effectively cutting it out of the background so the system can focus on the product itself. Some methods use text prompts to tell the computer exactly which object to find, while others rely on visual patterns to group parts together. This step is crucial because if the system looks at the wrong part of the image, it will miss the defect entirely.
Once the object is isolated, the system moves to the second stage: finding structural defects. These are the classic physical flaws like scratches, dents, or stains. Here, the researchers found that the most successful approaches do not try to teach the model what a defect looks like from scratch. Instead, they adapt the model's existing understanding of the world. For example, some methods use text prompts to describe a "perfect screw" and a "scratched screw," guiding the model to compare the image against these descriptions. Others use the model's internal attention mechanisms to spot areas that look different from the rest of the object. The survey notes that while these methods are powerful, they often struggle with very subtle defects that are hard to distinguish from normal variations in the material. The best results come from combining different techniques: using one model to find the object, another to describe it, and a third to spot the tiny irregularities.
The third and most complex stage involves logical anomalies. These are not physical flaws but errors in how things are put together. A logical defect might be a missing component, a part installed in the wrong order, or the wrong number of screws in a row. In these cases, the object parts might look perfectly normal on their own; the error is in the relationship between them. The survey explains that solving this requires a different kind of reasoning. Some systems use large language models to read a set of rules—like "there must be four bolts, and they must be arranged in a square"—and then check the image to see if the rules are followed. Other methods break the image down into individual parts and check if they match a known pattern. The researchers found that while these systems are getting better, they still struggle with complex scenes where parts are crowded together or where the rules are not clearly defined. The biggest hurdle remains the need for precise descriptions of what "correct" looks like, which can be difficult to write down for every possible product.
Despite these advances, the authors emphasize that the field is far from solved. A major issue is that many studies report their results in ways that make fair comparison impossible. Some papers claim to be "zero-shot" but actually use a few reference images, while others hide the fact that they used thousands of external images to train their models. The survey calls for a new standard in how these systems are tested, where researchers must clearly state exactly what data they had and how they used it. Without this transparency, it is difficult to know if a new method is a genuine breakthrough or just a clever trick that works only in a specific lab setting.
The survey also points out significant practical challenges for real-world deployment. Many of these advanced systems are slow and require powerful computers, which can be a problem for factories that need to inspect thousands of items per minute. Furthermore, the models are often trained on standard, low-resolution images, while factory cameras capture high-resolution details where the smallest defects hide. Resizing these images to fit the models can blur the very details the system needs to see. The researchers suggest that future work must focus on making these systems faster, more efficient, and better at handling the high-resolution, complex images found in actual production lines. They also highlight the need for systems that can learn continuously, adapting to new products and changing conditions without needing to be completely retrained from scratch.
Ultimately, this survey serves as a necessary reality check for a field that is moving very fast. Foundation models have opened the door to solving the cold-start problem, offering a way to inspect new products without needing massive datasets of broken examples. However, the path to reliable, industrial-scale deployment is not a straight line. It requires careful separation of what the model knows from what the factory provides, a clear understanding of the difference between physical and logical defects, and a commitment to rigorous, transparent testing. The researchers conclude that the next step is not just building smarter models, but building better ways to evaluate them, ensuring that the promise of artificial intelligence in manufacturing becomes a practical reality rather than just a theoretical possibility.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.