Modelling and Simulation of Neuromorphic Datasets for Anomaly Detection in Computer Vision

To address the scarcity of Dynamic Vision Sensor data, this paper introduces ANTShapes, a Unity-based simulation framework that generates customizable neuromorphic datasets with statistically labeled anomalies for training event-based computer vision models.

The Gist

Imagine you are trying to teach a robot how to spot a "weirdo" in a crowd. Maybe the robot is a security camera, and it needs to know when someone is running the wrong way, dancing in a straight line, or floating when they should be walking.

To teach this robot, you need a lot of practice videos. But here's the problem: the special cameras these robots use (called Neuromorphic Cameras or DVS) are rare, expensive, and hard to find. They don't work like normal cameras that take photos; instead, they only "see" changes in light, like a super-fast, ultra-efficient eye that only blinks when something moves. Because there are so few real videos from these cameras, researchers are stuck trying to teach their robots with very little data.

Enter ANTShapes: The "Digital Playground"

This paper introduces a new tool called ANTShapes. Think of it as a video game engine (built on the popular Unity platform) designed specifically to create fake but realistic training data for these special cameras.

Here is how it works, broken down into simple concepts:

1. The "Crowd" of Shapes

Instead of filming real people or cars, ANTShapes fills a virtual room with simple 3D shapes—cubes, spheres, pyramids, and even a monkey head (a famous 3D model).

• The Normal Crowd: Most of these shapes move in a predictable, "boring" way. They drift slowly, spin gently, or stay still. This represents the "normal" behavior of a crowd.
• The Anomalies: The tool randomly picks a few shapes to act "crazy." Maybe one cube is spinning 10 times faster than the rest, or a sphere is zooming across the room when everyone else is standing still. These are the "anomalies" the robot needs to learn to spot.

2. The "Math Magic" (The Bell Curve)

How does the tool decide what "normal" looks like? It uses a concept called the Central Limit Theorem.

• The Analogy: Imagine a classroom where every student is asked to walk across the room. Most will walk at a medium pace. A few will walk a bit faster, and a few a bit slower. If you graph their speeds, you get a "Bell Curve" (a hill shape).
• The Application: ANTShapes uses this math to generate the "normal" crowd. It creates a bell curve of behaviors. If a shape's behavior falls way out on the edge of the curve (like a student sprinting at 100mph), the tool marks it as an Anomaly (colored red in the simulation).

3. The "Special Eye" (Event-Based Vision)

This is the most important part. ANTShapes doesn't just make a video file like a movie. It simulates how a Neuromorphic Camera sees the world.

• Normal Camera: Takes a picture every second, capturing everything (even if nothing is moving). It's like taking a photo of a sleeping baby every minute.
• Neuromorphic Camera: Only "blinks" when something changes. If a shape moves, it sends a signal. If it stops, it goes silent.
• The Result: ANTShapes converts the 3D shapes into a stream of "spikes" or "blips" (black, white, and grey pixels). This is the exact format the AI needs to learn from, without needing a real, expensive camera.

4. Why is this a Big Deal?

Before this tool, researchers had two bad options:

1. Wait for real data: Hope someone films a weird event with a rare camera (which takes forever).
2. Fake it with old cameras: Try to turn normal video into neuromorphic data, which often looks glitchy and unreliable.

ANTShapes is the "Sandbox." It lets researchers:

• Create thousands of "what-if" scenarios instantly.
• Control exactly what counts as "weird" (e.g., "I only want to detect fast movement, ignore spinning").
• Test their AI models in a perfectly controlled environment before trying them on real streets.

The Catch (What's Missing?)

The authors are honest about the tool's limits. Right now, the "shapes" in the simulation can walk through each other like ghosts because the math assumes everything is independent. In the real world, people bump into each other; they have cause and effect.

The Future: The team plans to upgrade the tool to include "realistic" crowds (human-shaped agents) and cause-and-effect logic, so the simulation feels even more like a real city street.

In a Nutshell

ANTShapes is a digital toy box that lets scientists build their own "weirdness" in a controlled world. It generates the specific type of data that next-generation, low-power AI cameras need to learn how to spot trouble, solving the problem of not having enough real-world data to train them.

Technical Summary

1. Problem Statement

The research addresses a critical bottleneck in the field of neuromorphic computer vision: the scarcity of reliable Dynamic Vision Sensor (DVS) datasets.

• Hardware Limitations: While DVS cameras offer low power consumption, high dynamic range, and high capture rates suitable for edge devices, they are not widely deployed as security cameras. Consequently, real-world event-based data for anomaly detection is rare.
• Limitations of Existing Simulators:
  • Frame-to-Event Conversion: Methods that interpolate between standard video frames often fail if the source footage has a low temporal capture rate, leading to unreliable event data.
  • Physical Capture: Pointing a DVS camera at a monitor introduces external noise and is imperfect.
  • Existing 3D Simulators (e.g., CARLA): While capable of simulating 3D environments, tools like CARLA are designed for autonomous vehicles (ego-motion) rather than fixed surveillance. They lack built-in mechanisms for automated anomaly generation, requiring manual, labor-intensive labeling by users.
• The Gap: There is a lack of a comprehensive, parameterizable simulator that can generate bespoke neuromorphic datasets specifically for training Spiking Neural Networks (SNNs) in anomaly detection scenarios.

2. Methodology: ANTShapes

The authors introduce ANTShapes (Anomalous Neuromorphic Tool for Shapes), a novel dataset simulation framework built on the Unity engine. The system simulates abstract 3D scenes populated by objects with randomly generated behaviors to approximate the view of a fixed CCTV camera.

A. Anomaly Definition and Statistical Modeling

The core innovation lies in how "anomalies" are mathematically defined and generated using statistical principles:

• Agent Behaviors: Agents (3D objects) are parameterized by behaviors such as translation, rotation, scale, and texture. These behaviors are modeled using Normal Distributions ($N(\mu, \sigma)$).
• Central Limit Theorem Application: The system assumes that in a crowd of agents, common behaviors will tend toward a normal distribution.
• Likelihood Calculation:
  • The likelihood of an agent's behavior is calculated using the Cumulative Distribution Function (CDF) and transformed via a Chi-squared distribution to produce a normalized P-value ($\bar{r}_n$).
  • An agent is flagged as an anomaly if its overall likelihood ($\omega_n$) falls below a specific threshold ($v$).
• Crowd Behavior Simulation: To simulate "crowd-like" behavior where anomalies stand out, the system employs a cubic curve function (Eq. 5). This function shifts normally sampled values to center around a mean, creating a steep separation between normal and anomalous behaviors. A "fuzziness" coefficient allows users to smooth the boundary between normal and anomalous states.

B. Dataset Generation and Event Simulation

• Scene Configuration: Users can define:
  • Object Classes: 12 distinct shapes ranging from simple geometries (cubes, spheres) to complex models (Utah teapot, Suzanne monkey).
  • Behavioral Parameters: Independent control over mean ($\mu$) and standard deviation ($\sigma$) for X, Y, and Z axes regarding translation, rotation, and scale.
  • Visual Fidelity: Adjustable Gaussian noise, depth fog, field-of-view, and surface texture noise to mitigate mesh artifacts.
• Event Generation (Spiking):
  • The simulator calculates "spiking events" by comparing the monochromatic pixel intensity of the current frame ($f_t$) against the previous frame ($f_{t-1}$).
  • Bipolar Hysteresis Gating: A threshold ($\pm\beta$) is applied to the difference matrix.
    • Brightness increase $> \beta \rightarrow$ Event value 1.
    • Brightness decrease $< -\beta \rightarrow$ Event value 0.
    • No significant change $\rightarrow$ Event value 0.5.
• Temporal Control: The simulation runs with a default 1ms frame interval but can be adjusted to microsecond scales to better mimic the high-speed nature of real event-based sensors.

3. Key Contributions

1. ANTShapes Framework: The first dedicated simulator for generating abstract, configurable 3D neuromorphic datasets specifically tailored for anomaly detection in fixed-camera surveillance scenarios.
2. Statistical Anomaly Modeling: A novel approach to defining anomalies not as hard-coded rules, but as statistical deviations from a learned crowd distribution using the Central Limit Theorem and Chi-squared transformations.
3. Parameterization and Flexibility: The ability to generate datasets with an arbitrary number of samples by adjusting a limited set of parameters (e.g., $\mu$, $\sigma$, threshold $v$), allowing researchers to target specific agent properties for anomaly labeling.
4. Open-Source Availability: The tool is released as open-source software (Unity 6.1 project) with builds for Linux and Windows, accessible via GitHub.

4. Results and Current Status

• Prototype Status: At the time of writing, ANTShapes is in the early stages of development.
• Validation: The paper presents visual examples (screenshots and event matrices) demonstrating the generation of normal and anomalous behaviors (e.g., objects moving too fast or rotating excessively).
• Limitations Acknowledged:
  • Independence Assumption: The current model assumes events are non-causal and independent (agents can intersect), which is a simplification of real-world physics.
  • Lack of Extensive SNN Testing: The synthesized datasets have not yet been extensively tested against various Spiking Neural Network architectures to fully establish the tool's efficacy in practice.
  • Abstract Nature: The scenes are abstract (geometric shapes) rather than realistic urban environments, intended for low-level analysis of SNN learning characteristics first.

5. Significance and Future Work

• Enabling Edge AI Research: By providing a scalable source of training data, ANTShapes facilitates the development of SNNs for low-power, edge-based anomaly detection, a field currently hindered by data scarcity.
• Future Directions:
  • Realistic Scenes: Extending the simulator to include human-shaped agents and procedural urban environments.
  • Causality: Incorporating causal behavior (e.g., crowd flow, collision avoidance) to make the simulation more physically accurate.
  • Stereoscopic Vision: Adding binocular vision capabilities to better encode depth information.
  • Dynamic Cameras: Moving beyond static cameras to simulate ego-motion.

In summary, ANTShapes fills a vital gap in neuromorphic research by providing a flexible, statistically rigorous tool for generating synthetic event-based data, enabling the training and evaluation of SNNs for anomaly detection in scenarios where real-world data is unavailable.