Anomaly detection for generic failure monitoring in robotic assembly, screwing and manipulation

1. Problem Statement

Robotic manipulation tasks often encounter out-of-distribution (OOD) states due to environmental variations, task changes, or unexpected disturbances. These states can lead to unpredictable robot behavior, task failure, or physical damage. While Anomaly Detection (AD) can trigger failsafe behaviors, existing solutions face significant limitations:

• Lack of Generalizability: Most prior work focuses on specific tasks or control strategies, with little evidence of transferability across different robot controllers (e.g., position vs. impedance control) or task types.
• Data Efficiency: Many methods require extensive failure data for training, which is difficult to collect in industrial settings.
• Subtle Failures: Existing datasets often lack "subtle" failures (e.g., slight misalignments) that produce low-magnitude signal deviations, focusing instead on catastrophic failures.
• Contact-Rich Tasks: There is a scarcity of AD frameworks validated on complex, contact-rich industrial tasks (insertion, screwing, polishing) where force/torque signals are critical.

2. Methodology

The authors propose a data-driven, unsupervised anomaly detection framework based on Autoencoders (AE) that operates on multi-modal time-series data (force/torque, positions, velocities).

A. Data Processing

• Input: Multi-modal time-series data ($X$) including End-Effector (EE) pose, wrench forces/torques, joint velocities, and gripper states.
• Sliding Windows: Data is processed using sliding windows ($w_t$) with high temporal overlap to preserve fidelity.
• Normalization: The study tested mean/variance normalization but found no performance gain, opting to use unnormalized data to preserve raw signal characteristics.

B. Model Architecture

The study compares several Autoencoder-based architectures to find the most robust approach:

1. Variational Autoencoder (VAE): The primary model. It learns a probabilistic latent representation, minimizing a combined loss of reconstruction error and Kullback-Leibler (KL) divergence. This encourages a structured latent space.
2. Standard Autoencoder (AE): Optimizes only reconstruction loss.
3. Undercomplete Autoencoder (UAE): Features a latent dimension smaller than the input dimension.
4. SWCVAE: A VAE variant incorporating Convolutional Neural Network (CNN) layers to process input shapes before flattening.

C. Training and Inference

• Training: Models are trained exclusively on nominal (successful) task data. The goal is to learn the distribution of "normal" behavior.
• Reconstruction Error: During inference, the model attempts to reconstruct input windows. Anomalies are detected when the reconstruction error (Euclidean distance/L2-loss) exceeds a threshold.
• Thresholding Strategy: A novel, failure-data-free thresholding method is proposed. Thresholds are derived from the maximum reconstruction error observed across all nominal training windows, per feature. This avoids the need for labeled failure data during the thresholding phase.

D. Experimental Setup

The framework was validated on three distinct industrial tasks with different control strategies:

1. Cabling (Plug Insertion): Controlled via Diffusion Policy (Imitation Learning). Involves grasping a plug and inserting it into a connector.
2. Screwing: Controlled via Position Control. Involves picking, placing, and tightening screws.
3. Polishing: Controlled via Force/Impedance Control. Involves polishing aluminum parts with a sanding tool.

3. Key Contributions

• Cross-Task Generalization: Demonstrates that a single AD architecture (VAE) can be applied across diverse tasks (insertion, screwing, polishing) and control paradigms (IL, Position, Force) without task-specific re-engineering.
• Subtle Failure Detection: The dataset includes subtle failure modes (e.g., misaligned plugs, missing threads) characterized by low-magnitude force deviations, challenging the limits of current AD systems.
• Data Efficiency Analysis: Systematically evaluates how the volume of training data (number of demonstrations) impacts detection performance, finding that 50% of available data often yields optimal results.
• Novel Thresholding: Introduces a method to set detection thresholds based solely on nominal data statistics, eliminating the need for pre-collected failure data for calibration.

4. Experimental Results

The study evaluated performance using AUROC (Area Under the Receiver Operating Characteristic curve), F1-score, and detection latency.

• Cabling Task:
  • High Performance: Achieved near-perfect detection (AUROC > 0.98) for human disturbances, missing plugs, and misaligned plugs.
  • Challenge: "Incorrect Target Pose" was harder to detect (AUROC 0.78), prompting an ablation study showing that larger window sizes (500 steps) and full feature sets improve performance.
• Screwing Task:
  • High Performance: Reliable detection (AUROC > 0.96) for obstructed holes and misaligned screwdrivers.
  • Challenge: "Missing Screw" (no object at pickup) was difficult to detect (AUROC 0.52), likely due to a lack of force interaction to signal the anomaly.
• Polishing Task:
  • Mixed Results: Only severe failures (Human Disturbance, AUROC 0.93) were reliably detected.
  • Limitation: Subtle failures (surface roughness changes, wrong sandpaper) were undetected (AUROC < 0.4). The authors attribute this to the high natural variance in nominal force signals during polishing, which masks subtle anomalies.
• Data Efficiency: The model achieved peak AUROC (0.86) using only 50% of the training demonstrations (10 out of 20). Using the full dataset slightly reduced performance, suggesting that excessive variance in training data can introduce false positives.
• Latency: Detection latency was generally low (often < 3ms for severe failures), though some false positives occurred due to start-pose variations.

5. Significance and Conclusion

• Industrial Applicability: The work validates that unsupervised AD can be integrated into complex, real-world robotic workflows without requiring extensive failure datasets for training.
• Safety and Robustness: By reliably detecting OOD states, the system enables the triggering of failsafe behaviors (e.g., stopping the robot), which is crucial for deploying ML-based controllers (like Diffusion Policies) in safety-critical environments.
• Limitations & Future Work: The study highlights that AD performance is inversely proportional to the variance of the nominal task. Tasks with high inherent signal variance (like polishing) are harder to monitor for subtle defects. Future work should focus on stage-dependent thresholds (adjusting sensitivity based on the phase of the task) and improving the detection of subtle, low-force deviations.

In summary, this paper provides a robust, transferable framework for monitoring robotic failures, proving that a single VAE-based approach can effectively handle diverse industrial tasks, provided the anomaly causes a significant enough deviation from the learned nominal distribution.