Weakly Supervised Video Anomaly Detection Using CNN-LSTM with Confidence-Aware Automated Pseudo Label Generation

Surveillance cameras watch over our cities, stores, and streets, capturing a continuous stream of human activity. For decades, the goal has been to build computers that can watch this footage alongside us, spotting trouble the moment it happens. The challenge, however, is that teaching a computer to recognize a fight, a theft, or a fire is incredibly difficult. It requires showing the machine thousands of examples of these events, with every single moment of danger marked by a human expert. This manual labeling is slow, expensive, and often impossible to do for the vast amounts of video data generated every day. Without these detailed labels, standard computer programs struggle to learn what is normal and what is not, often getting confused by the sheer variety of everyday life.

To solve this, researchers at Shiraz University have developed a new way to teach computers to spot anomalies without needing a human to label every second of video. Instead of relying on expensive manual notes, their system builds its own understanding of what is happening by watching the video through three different lenses. First, it looks for movement, identifying any part of the screen where pixels are changing. Next, it tries to recognize specific objects, like a person or a vehicle, within those moving areas. Finally, it draws a precise outline around those objects to see exactly how much space they occupy. By combining these three observations, the system generates its own "guesses" about which parts of a video are normal and which are strange. These guesses, known as pseudo labels, are not perfect, but the researchers designed a special method to weigh how confident the system is in each guess, ignoring the uncertain ones and focusing on the clear signals.

The core of this new approach is a lightweight computer model that learns from these automated guesses. The researchers used a pre-trained vision system, which acts like a pair of eyes that has already learned to recognize shapes and objects from millions of images, and froze its settings so it would not change during training. This saved a massive amount of computing power, reducing the number of adjustable parts in the system by nearly ninety-four percent. The model then used a memory component to remember the sequence of events, connecting the dots between frames to understand how a scene evolves over time. To ensure the system did not jump erratically between labeling a scene as normal and then abnormal, they added a rule that encouraged smooth, consistent predictions from one moment to the next.

When tested on a large collection of surveillance videos containing thirteen different types of crimes, the system performed with a high degree of reliability. It correctly identified abnormal events in about eighty percent of the test cases and achieved a balanced score of seventy-eight percent in distinguishing between normal and abnormal activity. The researchers ran the experiment five times with different random starting points to ensure the results were consistent, and the performance remained steady across all trials. Crucially, the system achieved this without any human having to manually mark the dangerous moments in the training videos. The automated process of detecting motion, identifying objects, and outlining them proved to be a viable substitute for human annotation, making it possible to scale up video monitoring to much larger datasets than ever before.

The study also revealed that each part of the automated pipeline was essential. When the researchers removed the motion detection, the object recognition, or the precise outlining, the system's ability to spot trouble dropped significantly. This confirmed that the combination of these three visual cues provided a richer, more reliable picture than any single method could offer on its own. While the system is not yet perfect and can still struggle in very crowded scenes or under poor lighting, it represents a significant step forward in making automated surveillance practical. By removing the bottleneck of manual labeling and drastically cutting the computing cost, this approach offers a realistic path toward deploying intelligent monitoring systems in the real world, where they can watch over public spaces with a level of attention that humans simply cannot maintain.