UAV-SEAD: State Estimation Anomaly Dataset for UAVs

1. Problem Statement

Unmanned Aerial Vehicles (UAVs) rely heavily on accurate state estimation (position, orientation, and velocity) for safe and reliable operation. Anomalies in state estimation—deviations between the estimated state and the true system behavior—can lead to catastrophic mission failures or safety hazards.

• The Gap: Existing research on anomaly detection often relies on:
  • Simulated data: Which may not capture real-world complexities.
  • Injected faults: Artificially induced hardware failures (e.g., broken propellers, sensor spoofing) that do not reflect naturally occurring anomalies.
  • Limited scope: Many datasets focus on specific actuator or sensor failures rather than the holistic state estimation process.
• The Challenge: Detecting state estimation anomalies is difficult because they often arise from complex interactions between sensors (IMU, GPS, Barometer, Visual Odometry), the estimator (EKF), and the environment. Furthermore, collecting real-world anomalous data is dangerous and time-consuming, leading to a scarcity of high-quality, naturally occurring anomaly datasets.

2. Methodology

The authors developed a comprehensive framework for data collection, classification, and annotation to create the UAV-SEAD dataset.

• Data Collection Environment:

  • Platform: Custom-designed quadrotors equipped with PX4 autopilot firmware (running on Pixhawk 2 Cube and Pixhawk 4 Mini).
  • Hardware Diversity: Six distinct UAV configurations with varying sensor suites, including IMUs, GPS, barometers, magnetometers, distance sensors (LiDAR/Ultrasonic), optical flow sensors, and visual odometry (via companion computers running SLAM).
  • Environments: Data was collected in diverse indoor (warehouses, sports halls) and outdoor settings to capture a wide range of environmental disturbances.
  • Natural Anomalies: Crucially, no artificial faults were injected. Anomalies were captured naturally during the development, testing, and operational phases of the UAVs.

• Anomaly Classification Framework:
  The authors propose a structured taxonomy categorizing state estimation anomalies into four primary classes based on their source and manifestation:

  1. Mechanical and Electrical Anomalies: Caused by vibration, sensor misalignment, center-of-mass shifts, motor failures, or power fluctuations (voltage drops/current spikes).
  2. External Position Anomalies: Errors in position estimates derived from external sources like Visual Odometry (VO) or Optical Flow, often due to lighting conditions, feature loss, or algorithmic failures.
  3. Global Position Anomalies: Errors related to GPS, including signal loss, multipath effects, spoofing, or sudden coordinate jumps.
  4. Altitude Anomalies: Discrepancies in height estimation caused by barometric pressure changes, distance sensor errors (e.g., on reflective surfaces), or EKF filtering issues.

• Annotation Process:

  • Human-in-the-Loop: Since no single sensor can serve as absolute ground truth in all scenarios, expert human annotators analyzed flight logs to identify divergences between estimated states and observed behaviors.
  • Tools: A custom annotation toolbox was developed to visualize time-series data (position, thrust, sensor readings) and label anomalous segments.
  • Anomaly Types: The dataset captures Outlier (point), Contextual (invalid within context), and Collective (group of normal points forming an anomaly) anomalies.

3. Key Contributions

• UAV-SEAD Dataset: The largest real-world UAV dataset specifically for state estimation anomaly detection to date.
  • Scale: 1,396 flight logs totaling over 52 hours of flight time.
  • Authenticity: Contains zero injected faults; all anomalies occurred naturally.
  • Richness: Includes multivariate sensor streams (IMU, GPS, Barometer, Magnetometer, Distance, Visual Odometry, Optical Flow) and autopilot outputs (EKF estimates, actuator controls).
• Structured Taxonomy: A novel classification system for state estimation anomalies that moves beyond simple sensor faults to categorize complex system-level behaviors (Mechanical/Electrical, External Position, Global Position, Altitude).
• Open Source Resources: The dataset and the custom annotation toolbox are publicly available (hosted on Hugging Face), facilitating reproducibility and community expansion.
• Comparative Analysis: The paper provides a rigorous comparison with existing datasets (e.g., ALFA, UAV-FD, RflyMAD), highlighting that most existing datasets rely on simulations or injected hardware failures, whereas UAV-SEAD focuses on naturally occurring estimation errors.

4. Results and Statistics

• Dataset Composition:
  • Normal Flights: 900 flights (approx. 38.4 hours).
  • Anomalous Flights: 496 flights containing anomalies.
  • Anomaly Distribution:
    • External Position: The most frequent class (50.65% of annotated anomaly duration), often manifesting as drift or jumps in visual odometry.
    • Global Position: 23.92% of anomaly duration; characterized by sudden GPS jumps.
    • Altitude: 17.07% of anomaly duration; often transient.
    • Mechanical/Electrical: 8.36% of anomaly duration; linked to thrust saturation and motor issues.
• Case Studies: The paper presents detailed time-series visualizations (Figures 5–8) demonstrating how specific anomalies (e.g., a sudden reset of Y-position without attitude change, or altitude drift despite thrust changes) are identified through cross-validation of multiple sensor streams.
• Performance: The dataset successfully demonstrates that anomalies often require collective analysis of multiple sensors (e.g., comparing thrust commands against altitude estimates) rather than single-sensor thresholding.

5. Significance

• Bridging the Research Gap: UAV-SEAD addresses the critical lack of real-world data for training data-driven anomaly detection models (e.g., Deep Learning, Autoencoders).
• Safety and Reliability: By focusing on state estimation rather than just hardware failure, the dataset enables the development of systems that can detect subtle drifts or filter divergences before a catastrophic failure occurs.
• Generalizability: The classification framework is hardware-agnostic, making it applicable to various UAV platforms beyond the specific models used in the study.
• Future Impact: The dataset serves as a foundational benchmark for developing robust Fault Detection and Isolation (FDI) systems, ultimately contributing to safer autonomous aerial operations in complex, GPS-denied, or dynamic environments.