From Offline to Periodic Adaptation for Pose-Based Shoplifting Detection in Real-world Retail Security

This paper presents a periodic adaptation framework for unsupervised, pose-based shoplifting detection on edge IoT devices, validated by a new large-scale real-world dataset (RetailS) and demonstrating superior performance and rapid update times compared to offline baselines.

The Gist

Imagine you own a busy grocery store. You have cameras everywhere, but you can't have a security guard staring at every screen 24/7. It's too expensive, and humans get tired. So, you install a smart computer system to watch the cameras for you.

The problem is that most of these smart systems are like students who only study for a final exam in a quiet library. They learn the rules once, take a test, and then stop learning. But a real store is chaotic: the lighting changes, the shelves move, the crowd gets denser, and shoplifters get smarter. A "library student" system gets confused by the real world and starts crying wolf (false alarms) or missing the actual thefts.

This paper introduces a new way to build these security systems, using a "Periodic Adaptation" framework. Here is how it works, broken down into simple concepts:

1. The "Skeleton" Trick (Privacy & Speed)

Instead of recording the actual video of people (which looks like a movie and raises privacy concerns), the system turns people into moving stick figures (skeletons).

• The Analogy: Imagine a dance instructor who only cares about how you move your arms and legs, not what you are wearing or what your face looks like.
• Why it helps: It's much faster for the computer to process a stick figure than a high-definition video. It also protects customer privacy because you can't identify who the person is, only what they are doing.

2. The "Smart Filter" (The Gatekeeper)

The system doesn't try to learn from everything it sees. That would be like trying to learn a new language by reading every book in a library at once.

• The Analogy: Think of a bouncer at a club. The bouncer (the AI) watches the crowd. If someone looks suspicious, the bouncer flags them. If they look normal, the bouncer lets them pass.
• The Twist: The system is smart enough to know that "normal" behavior changes. Sometimes people walk fast; sometimes they stand still. The system uses a special "filter" to decide which normal-looking moments are safe to save for later study.

3. The "Night School" (Periodic Adaptation)

This is the most important part. Instead of the system being static, it has a night school schedule.

• The Analogy: Imagine the security guard works during the day. At night, while the store is closed, the guard takes the "safe" moments they saved during the day and studies them. They update their mental rulebook to say, "Oh, I see that people in this aisle walk differently now because of the new display."
• How it works: Every 12 or 24 hours, the system takes the data it collected, runs a quick training session (like a 30-minute cram session), and updates its brain. Then, it goes back to work the next day with a fresh, smarter perspective.

4. The "RetailS" Dataset (The Practice Ground)

To make sure this actually works, the researchers didn't just use fake computer simulations. They went to a real store and built a massive new dataset called RetailS.

• The Analogy: Most previous studies were like training a pilot in a flight simulator with perfect weather. This team built a dataset that includes real turbulence, rain, and unexpected passengers.
• What's in it: They have thousands of hours of normal shopping, plus real shoplifting incidents caught on camera, and even "staged" thefts where researchers pretended to steal things to test the system. This ensures the AI is ready for the messy reality of a real store.

5. The Results: Why It Matters

The researchers tested their "night school" system against the old "library student" systems.

• The Outcome: The new system was 91.6% better at catching shoplifters without getting confused by normal changes in the store.
• The Speed: The "night school" session only took about 30 minutes on a standard computer, meaning it's fast enough to be used in real stores without needing a supercomputer.

The Big Picture

This paper solves a major problem in AI: How do you make a robot smart enough to handle a changing world without needing a human to reprogram it every day?

By using stick figures for privacy, smart filters to pick good data, and a nightly study session to keep learning, they created a security system that is:

1. Privacy-friendly (no faces, just skeletons).
2. Self-updating (gets smarter every day).
3. Real-world ready (tested in a real store, not just a lab).

It's like upgrading your store security from a static statue that never moves, to a living, learning guard that adapts to the store every single night.

Technical Summary

1. Problem Statement

Shoplifting represents a massive economic challenge for retailers, with incidents rising significantly and losses projected to exceed $53 billion by 2027. Despite extensive video surveillance, continuous human monitoring is infeasible, and existing automated solutions face critical limitations in real-world Internet of Things (IoT) deployments:

• Static Models: Most current Video Anomaly Detection (VAD) systems rely on offline training with fixed thresholds, failing to adapt to "domain drift" (changes in store layout, lighting, camera angles, and shopper behavior).
• Privacy and Resource Constraints: Pixel-based approaches raise privacy concerns and are computationally too heavy for edge devices.
• Data Scarcity: There is a lack of large-scale, realistic, multi-camera datasets designed specifically for IoT streaming scenarios.
• Deployment Gap: Existing methods are often validated in controlled lab environments, leading to high false-positive rates and poor scalability when deployed in dynamic retail settings.

2. Methodology

The authors propose a periodic adaptation framework designed for edge-cloud IoT architectures, shifting from static offline evaluation to dynamic, continual unsupervised learning.

A. Pose-Based Representation

Instead of using raw RGB pixels, the system utilizes 2D pose keypoints.

• Privacy: Removes identifiable facial/body features.
• Efficiency: Skeletal sequences are compact, reducing bandwidth and computational load on edge devices.
• Robustness: Less sensitive to lighting changes and visual noise compared to pixel-based methods.

B. The Periodic Adaptation Pipeline

The framework operates in a three-stage loop (Filtering $\rightarrow$ Collection $\rightarrow$ Training):

1. Filtering (Edge): A pretrained model processes streaming video from distributed cameras. It generates anomaly scores and uses adaptive thresholds to filter frames.
   • Thresholding Strategy: The paper introduces HPRS (Harmonic Mean of Precision, Recall, and Specificity) as a metric for threshold selection. Unlike F1-score, HPRS explicitly penalizes False Positives (specificity), which is critical for preventing alert fatigue in retail.
   • Pseudo-Labeling: Frames with low anomaly scores are treated as "normal" and selected for the training buffer.
2. Collection (Edge/Cloud): Filtered frames are aggregated across all cameras in time-sliced windows (e.g., half-day or full-day). This prevents bias toward high-traffic cameras and ensures a representative sample of store-wide activity.
3. Training (Back-end): When a buffer reaches a preset size, the model is fine-tuned on the accumulated pseudo-labeled data.
   • Injection: To prevent the model from forgetting anomalies, curated anomaly samples are injected into the training batch at a 9:1 normal-to-abnormal ratio.
   • Update Cycle: Updated model weights are pushed back to the edge devices. Crucially, thresholds remain fixed (calibrated offline) to ensure stable inference, while only the model weights adapt.

C. Dataset: RetailS

To support this research, the authors introduced RetailS, a new large-scale dataset:

• Scale: Contains ~20 million frames of normal shopping behavior and thousands of shoplifting incidents.
• Composition:
  • Real-world Test Set: Authentic shoplifting incidents extracted from 2 years of store surveillance logs.
  • Staged Test Set: Controlled scenarios covering 5 concealment strategies (hiding in pants, hoodie pockets, bags, under jackets) across 15 different shelf locations.
  • Multi-camera: Captured via 6 cameras at 15 FPS, reflecting heterogeneous IoT environments.
• Privacy: All data is anonymized into pose sequences (17 keypoints per person).

3. Key Contributions

1. Periodic Adaptation Framework: A novel IoT-oriented pipeline that enables continual, unsupervised learning on edge devices, addressing the limitations of static offline models.
2. RetailS Dataset: The first large-scale, multi-camera, pose-based dataset specifically designed for IoT streaming analysis, bridging the gap between lab benchmarks and real-world retail operations.
3. HPRS Thresholding: Introduction of the HPRS metric for threshold selection, demonstrating superior control over false alarms compared to standard F1-score optimization in imbalanced retail scenarios.
4. Comprehensive Benchmarking: A rigorous evaluation of state-of-the-art pose-based models (STG-NF, SPARTA, TSGAD) under both offline and periodic adaptation protocols, including time-efficiency analysis on edge-grade hardware.

4. Results

The experiments were conducted on three models: STG-NF, SPARTA, and TSGAD.

• Performance Improvement: The periodic adaptation framework outperformed offline baselines in 91.6% of evaluations (AUC-ROC and AUC-PR).
• Thresholding Efficacy: Thresholds selected via HPRS outperformed F1-based thresholds in 9 out of 12 cases, significantly reducing false positives while maintaining detection sensitivity.
• Update Frequency: Half-day update cycles (approx. 6 hours) yielded better performance than daily updates, indicating that more frequent adaptation better captures local behavioral drift.
• Efficiency:
  • Lightweight models (SPARTA, STG-NF) completed training updates in under 10 minutes on edge-grade hardware.
  • Heavier models (TSGAD) took up to 27 minutes for half-day updates, highlighting the trade-off between model complexity and IoT feasibility.
• Generalization: Models trained on the diverse RetailS dataset showed significantly better generalization to real-world test sets compared to those trained on smaller, staged datasets (like PoseLift).

5. Significance and Impact

This paper addresses a critical gap in retail security by moving beyond theoretical benchmarks to deployable IoT solutions.

• Operational Feasibility: It demonstrates that periodic adaptation can run efficiently on edge hardware, making real-time, self-improving security systems viable for retailers.
• Privacy-Preserving: By relying on pose data, the solution aligns with increasing global privacy regulations while maintaining high detection accuracy.
• Robustness to Drift: The study proves that static models degrade quickly in dynamic environments, whereas periodic adaptation maintains high performance despite changes in store layout, lighting, and shopper behavior.
• Practical Guidelines: The authors provide actionable insights for deployment, such as the superiority of HPRS thresholds for false-alarm control and the necessity of cross-camera data collection to avoid spatial bias.

In conclusion, the paper establishes a new standard for smart retail security, proving that combining pose-based representations with periodic, unsupervised adaptation creates a scalable, privacy-preserving, and highly effective system for detecting shoplifting in real-world IoT environments.