⚡ electrical engineering

Relative State Estimation using Event-Based Propeller Sensing

This paper presents a decentralized relative state estimation framework for quadrotor swarms that leverages event cameras to track propeller motion, extracting frequency and orientation data from real-world outdoor flights to overcome the latency, scale ambiguity, and visual limitations of traditional monocular camera methods.

Original authors: Ravi Kumar Thakur, Luis Granados Segura, Jan Klivan, Radim Špetlík, Tobiáš Vinklárek, Matouš Vrba, Martin Saska

Published 2026-04-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ravi Kumar Thakur, Luis Granados Segura, Jan Klivan, Radim Špetlík, Tobiáš Vinklárek, Matouš Vrba, Martin Saska

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to guess the speed and direction of a friend flying a drone right in front of you, but you can't talk to them, and the wind is blowing so hard that your eyes are getting blurry.

This is the problem this paper solves. It teaches a swarm of drones how to "see" and understand each other without needing a perfect camera or a radio connection.

Here is the breakdown of their clever solution, using simple analogies:

1. The Problem: The "Blurry Snapshot" Camera

Most drones use standard cameras, which are like taking a photo 30 times a second. If the drone spins its propellers super fast, or if the sun is glaring, or if there is motion blur, these cameras get confused. They are like a photographer trying to take a picture of a hummingbird's wings; the result is just a blurry mess. They also struggle to tell how far away something is (the "scale" problem).

2. The Solution: The "Event Camera" (The Super-Sensitive Eye)

The authors use a special Event Camera. Think of this not as a camera that takes photos, but as a high-speed security guard who only blinks when something moves.

How it works: Instead of recording a full image, it only records tiny "events" (pixels changing brightness) with microsecond precision.
The Analogy: If a standard camera is a video recording, the event camera is a stream of text messages saying, "Something moved here at 10:00:01.0001!" It ignores the static background and focuses only on the motion. This makes it incredibly fast and immune to blinding light or motion blur.

3. The Secret Sauce: Listening to the "Hum"

The core idea is to look at the propellers (the spinning blades) of the other drone.

The Propeller as a Metronome: When a drone spins its propellers, it creates a rhythmic pattern of events in the camera, like a metronome ticking.
The Magic: By counting how fast these "ticks" happen, the observing drone can calculate exactly how fast the propellers are spinning (RPM).
Why does this matter? In physics, the speed of the propellers tells you how much thrust (upward force) the drone is generating. If you know the thrust, you can guess how fast the drone is accelerating up or down.

4. The "Two-Brain" System

The paper describes a system that uses two different "brains" (mathematical filters) working together:

Brain A (The Position Tracker): This brain looks at where the drone is in the camera and combines it with the thrust data derived from the propeller speed.
- Analogy: Imagine you are watching a car drive away. You can see where it is, but you don't know if it's speeding up or slowing down. But if you hear the engine revving (the propeller speed), you know exactly how hard it's pushing. This brain uses that "engine sound" to predict the drone's movement much better than just looking at it.
Brain B (The Tilt Tracker): This brain looks at the shape of the spinning propeller.
- Analogy: When a spinning coin is flat, it looks like a circle. When you tilt it, it looks like an oval (an ellipse). By measuring how "squashed" the spinning propeller looks, the system can figure out if the drone is tilting left, right, forward, or backward.

5. The Result: A Decentralized Swarm

The paper tested this on real drones flying outdoors.

The Test: One drone (the observer) watched another drone (the target) fly around, doing loops and side-steps.
The Outcome: The system estimated the other drone's speed and tilt with very high accuracy (less than 3% error on propeller speed).
Why it's a big deal: Usually, drones need to talk to each other via radio to know where they are. This system allows them to be "decentralized." They can fly in a swarm, avoid collisions, and stay in formation just by watching each other's propellers, even if their radios fail or the environment is chaotic.

Summary

Think of this technology as teaching drones to dance together by listening to the rhythm of their own shoes. Instead of needing a map or a phone call to coordinate, they just watch the spinning blades of their neighbors, count the beats, and instantly know where everyone is and where they are going. It turns a chaotic, noisy visual world into a precise, rhythmic conversation.

1. Problem Statement

Autonomous swarms of Multi-Unmanned Aerial Vehicles (UAVs) require accurate, low-latency relative state estimation to coordinate formation flight, avoid collisions, and maintain poses. Traditional approaches rely on frame-based monocular cameras, which suffer from:

High Latency and Low Temporal Resolution: Inability to capture fast dynamics.
Scale Ambiguity: Difficulty in determining absolute depth without additional sensors.
Environmental Sensitivity: Performance degradation under motion blur, extreme illumination changes, and background clutter.

Furthermore, reliance on explicit communication for state sharing introduces vulnerabilities to latency and packet loss. The paper addresses the need for a decentralized, vision-based relative localization system that operates robustly in challenging environments by leveraging event cameras (neuromorphic sensors) and exploiting the periodic motion of quadrotor propellers.

2. Methodology

The proposed framework estimates the relative state (position, velocity, roll, and pitch) of a target quadrotor using an observer quadrotor equipped with a downward-facing event camera. The system operates through a multi-stage pipeline:

A. Event Data Processing & Propeller Detection

Data Representation: The event camera outputs asynchronous streams of events $(x, y, t, p)$ triggered by logarithmic brightness changes.
Temporal Chunking: Data is processed in 10ms temporal chunks.
Detection: Two methods are implemented to identify propeller regions of interest (ROI):
1. Connected Components (CC): Accumulates events into a binary density map, applies morphological erosion, and labels blobs.
2. HDBSCAN Clustering: A density-based clustering approach that treats events as 2D points, suitable for non-circular appearances at oblique angles.
Tracking: Detected propellers are tracked across time using the Norfair framework, maintaining IDs even during brief occlusions via motion prediction.

B. Frequency Estimation (RPM)

Signal Extraction: Within the tracked propeller ROIs, events are temporally binned to create a signal representing blade passage rates.
Frequency Calculation: Fast Fourier Transform (FFT) is applied to the binned event counts to identify the dominant frequency peak.
Smoothing: A Kalman Filter (KF) smooths the raw frequency estimates to reject outliers and ensure temporal consistency.

C. Relative State Estimation (Coupled Kalman Filters)

The system employs two coupled Kalman Filters:

Orientation Estimation KF:
- Input: Differential propeller frequencies (Left vs. Right motors) and geometric primitives.
- Mechanism:
  - Roll/Pitch: Derived from the differential thrust of motors (Left-Right imbalance creates roll angular velocity).
  - Geometric Constraint: The propeller disk appears as an ellipse in the event stream. Using Perspective-1-Ellipse (P1E), the system back-projects the fitted ellipse to recover the body-frame $z$ -axis (thrust direction) in the world frame.
- Output: Estimates Roll ( $\phi$ ) and Pitch ( $\theta$ ). Note: Yaw is unobservable from the ellipse geometry alone.
Position Estimation KF:
- Input: Thrust magnitude (derived from total RPM squared) and camera-derived position.
- Dynamics Model: Uses a kinematic model where total thrust $T \propto \sum \Omega_i^2$ . The thrust vector is aligned with the estimated body $z$ -axis from the Orientation KF.
- Measurement Update: Monocular depth is estimated using the known physical distance between propellers and their pixel separation (stereo-like disparity).
- Output: Estimates 3D Position ( $p$ ) and Velocity ( $v$ ).

3. Key Contributions

Event-Based Propeller Sensing: A novel framework that treats propeller rotation not just as a visual feature, but as a direct source of kinematic data (thrust and angular velocity) for state estimation.
Decentralized Localization: Enables relative localization without explicit communication, relying solely on visual perception of the target's mechanical dynamics.
Robustness to Environment: Demonstrates that event cameras, with their microsecond resolution and high dynamic range, overcome the motion blur and lighting limitations of frame-based cameras in outdoor swarm scenarios.
Dual-Mode Detection: Validates that both Connected Components and HDBSCAN clustering yield nearly identical frequency estimation accuracy, allowing for a trade-off between computational cost (CC is faster) and geometric adaptability (HDBSCAN handles oblique angles better).

4. Experimental Results

The system was evaluated on five real-world outdoor flight sequences involving a leader-follower formation. Ground truth was provided by RTK-GNSS and IMU.

Frequency Estimation Accuracy:
- Achieved a Mean Absolute Percentage Error (MAPE) of ~2.7% for propeller RPM estimation.
- Both detection methods (CC and HDBSCAN) showed negligible difference in RPM accuracy, confirming that the frequency extraction pipeline is the dominant factor, not the spatial detection method.
State Estimation Performance:
- Position: Mean RMSE of 0.30m (x), 0.12m (y), and 0.90m (z). The higher error in the $z$ -axis is attributed to monocular depth uncertainty.
- Velocity: Mean RMSE below 0.5 m/s across all axes.
- Orientation: Mean RMSE of 5.72° (roll) and 9.18° (pitch). Roll estimation was more accurate than pitch due to the stronger observability of left-right motor differentials in the test maneuvers.
Robustness: The system maintained stable tracking even during aggressive lateral maneuvers, though performance degraded slightly when propellers were partially occluded or moved out of the frame.

5. Significance and Conclusion

This work demonstrates that event cameras can serve as the primary sensor for decentralized swarm coordination, effectively replacing or augmenting communication links. By extracting physical dynamics (RPM) directly from the visual stream, the system provides a "physics-informed" state estimate that is robust to visual clutter and high-speed motion.

The approach offers a pathway for resilient autonomous swarms in GPS-denied or communication-degraded environments (e.g., disaster relief, planetary exploration). Future work aims to extend this to multi-quadrotor scenarios and further refine the extraction of geometric primitives from periodic events.