Event-Only Drone Trajectory Forecasting with RPM-Modulated Kalman Filtering

This paper proposes a novel event-only drone trajectory forecasting method that extracts propeller rotational speed directly from raw event data and integrates it into an RPM-aware Kalman filter, achieving superior short-to-medium horizon prediction accuracy compared to learning-based approaches without relying on RGB imagery or training data.

The Gist

Imagine you are trying to predict where a hummingbird is going to fly next. If you use a regular camera (like the one in your phone), you might get a blurry picture because the bird is moving so fast. By the time the camera takes the next "snapshot," the bird has already changed direction. It's like trying to guess the path of a race car by looking at a series of still photos taken once every second; you'll miss all the sharp turns.

This paper introduces a smarter way to track fast-moving drones using a special kind of camera called an Event Camera, combined with a mathematical "guessing game" called a Kalman Filter.

Here is the breakdown of how it works, using simple analogies:

1. The Super-Sensitive Eye (Event Camera)

Think of a regular camera as a flipbook. It takes a full picture, waits a split second, takes another full picture, and so on. If something moves too fast between those pictures, it looks like a smear.

An Event Camera, however, is more like a room full of tiny, hyper-alert security guards. Each guard (pixel) only speaks up when they see something change. If a drone flies past, the guards don't take a photo of the whole room; they just shout, "Hey! Something moved here!" and "Hey! Something moved there!"

• Why it helps: Because they only react to changes, they don't get confused by motion blur. They can see a drone spinning its propellers at 10,000 miles per hour without the image getting fuzzy.

2. Listening to the Hum (RPM Extraction)

The secret sauce of this paper is that the researchers realized they could listen to the drone's "engine" just by watching these event signals.

• The Analogy: Imagine you are in a dark room and you can't see a ceiling fan, but you can hear it spinning. If the fan spins faster, the "whoosh" sound gets higher and faster.
• The Tech: The event camera sees the propeller blades flashing on and off. The computer counts these flashes to figure out exactly how fast the propellers are spinning (Revolutions Per Minute, or RPM).
• The Insight: If the propellers are spinning wildly fast, the drone is probably about to do a crazy, aggressive maneuver (like a sharp turn or a dive). If they are spinning slowly, the drone is likely just hovering or drifting gently.

3. The Smart Predictor (RPM-Modulated Kalman Filter)

Now, how do we predict where the drone goes next? The authors use a Kalman Filter, which is basically a very smart "best guess" calculator.

• The Old Way (Vanilla Filter): Imagine a weather forecaster who always assumes the wind will blow at the same speed. If a sudden storm hits, their prediction is way off because they didn't adjust for the change.
• The New Way (RPM-Modulated): This new system is like a weather forecaster who checks the wind speed right now.
  • Scenario A: The drone is hovering (slow RPM). The calculator says, "Okay, it's calm. I'm pretty sure it will keep going straight." It trusts its own motion model.
  • Scenario B: The drone's propellers suddenly spin up to max speed (high RPM). The calculator says, "Whoa! Things are getting chaotic! I shouldn't trust my 'straight line' guess as much. I need to pay extra attention to where the drone actually is right now because it might jerk around."

By adjusting its "confidence" based on the propeller speed, the system becomes much better at predicting sudden, jerky movements.

4. The Results: Beating the "AI" Giants

The researchers tested this on a massive dataset called FRED (which has hours of drone footage in rain, at night, and indoors).

• The Competition: They compared their method against fancy Deep Learning (AI) models. These AI models are like students who memorize thousands of past flight paths. They are great if the drone flies exactly like it did in the past, but they get confused if the drone does something new.
• The Winner: The "Event Camera + RPM + Smart Filter" method won. It was more accurate than the AI models, even though it didn't need to be "trained" on thousands of examples. It just used physics and math to figure it out in real-time.

Why This Matters

• No Blurry Mess: It works in the dark, in the rain, and when things are moving super fast.
• No Heavy Brains: It doesn't need a super-computer or a massive database of training data. It's lightweight and fast.
• Real-World Safety: This is crucial for things like anti-drone defense (stopping a rogue drone) or air traffic control (making sure a drone doesn't crash into a plane). If you can predict where a drone is going to be in the next half-second, you can avoid a collision.

In a nutshell: Instead of trying to memorize every possible flight path like a robot student, this method acts like a seasoned pilot who listens to the engine's hum to instantly know if the plane is about to be calm or crazy, and adjusts their prediction accordingly.

Technical Summary

1. Problem Statement

Trajectory forecasting for Unmanned Aerial Vehicles (UAVs) is critical for airspace monitoring, collision avoidance, and anti-drone technologies. However, existing methods face significant challenges:

• Limitations of RGB Cameras: Conventional cameras suffer from motion blur, limited frame rates, and latency, especially when tracking fast-moving or aggressively maneuvering drones. These issues degrade state observation accuracy.
• Limitations of Deep Learning (DL): While DL models (e.g., Transformers, LSTMs) can learn complex patterns, they require massive training datasets, high computational resources, and often fail to generalize to drone types or flight patterns not seen during training.
• The Gap: Event cameras offer high temporal resolution and no motion blur, making them ideal for high-speed vision. However, their potential for trajectory prediction remains underutilized. Previous attempts using event data with DL showed only modest improvements over non-event baselines.

Core Objective: To develop a robust, real-time, event-only trajectory forecasting method that does not rely on RGB imagery or large-scale training data, by leveraging physical cues (propeller rotation) extracted directly from event streams.

2. Methodology

The proposed framework is a non-learning-based approach that combines event camera data processing with a modified Kalman Filter. The pipeline consists of three main stages:

A. RPM Estimation from Event Data

The system extracts the propeller rotation speed (Revolutions Per Minute - RPM) directly from raw event data within the drone's bounding box.

1. Frequency Mapping: A spatial frequency map is constructed by counting events per pixel.
2. Segmentation: A percentile-based threshold (70th percentile) separates high-frequency propeller pixels from low-frequency airframe pixels.
3. Period Histogram: Timestamps of ON/OFF events for filtered propeller pixels are binned into a histogram (0–25.6ms range).
4. RPM Calculation: The dominant period ($\hat{T}$) is identified from the histogram peak. The frequency ($f = 1/\hat{T}$) is converted to RPM using the formula:
   $$RPM = f \cdot \frac{60}{N_b}$$
   (where $N_b$ is the number of blades).
   Note: The system adapts to lighting and speed changes via a First-In-First-Out (FIFO) queue that removes histogram entries older than 100ms.

B. RPM-Modulated Kalman Filtering

The trajectory prediction is formulated as a state estimation problem using a discrete-time constant velocity model. The state vector includes center position $(c_x, c_y)$ and velocity $(v_x, v_y)$.

• Standard KF: Uses a fixed process noise covariance matrix ($Q$).
• Proposed RPM-Modulated KF: The process noise matrix is dynamically scaled based on the estimated RPM.
  • Logic: High RPM correlates with aggressive maneuvers and rapid acceleration (high uncertainty). Low RPM correlates with hovering or smooth flight (low uncertainty).
  • Scaling Factor ($\alpha_v$): Defined as $\alpha_v = \max(0.5, 1 + 2r + \max(0, \dot{r}))$, where $r$ is the normalized RPM level and $\dot{r}$ is the rate of change.
  • Effect: When RPM is high or increasing, the filter increases motion uncertainty, placing more weight on the measurement update. When RPM is low, the filter relies more heavily on the motion model.

C. Forecasting

• The current state is estimated using the RPM-modulated Kalman Filter.
• Future states are extrapolated using the kinematic motion model for horizons of 0.4s and 0.8s.
• Intermediate poses are generated at 33ms timesteps for full trajectory evaluation.

3. Key Contributions

1. Event-Only RPM Extraction: A novel method to extract propeller rotation speed directly from raw event data without RGB imagery or training.
2. RPM-Modulated Process Noise: A Kalman Filter variant where the process noise matrix is dynamically adjusted based on detected propeller speed, allowing the estimator to adapt to the drone's maneuverability in real-time.
3. State-of-the-Art Performance: Achieved superior results on the FRED dataset compared to both Deep Learning baselines and standard Kalman Filters, proving that physical modeling with event data outperforms data-driven approaches in this specific domain.
4. Robustness: Demonstrated effectiveness in challenging conditions (rain, night, indoor, highly dynamic motion) where RGB cameras fail due to motion blur.

4. Experimental Results

The method was evaluated on the FRED dataset (the largest event-based drone dataset), comparing against:

• Deep Learning Baselines: LSTM, Transformer, and CNN+Transformer (using RGB, Event, or both).
• Classical Baselines: Linear Extrapolation and Vanilla Kalman Filter.

Key Metrics (Average Displacement Error - ADE / Final Displacement Error - FDE in pixels):

Method	Input	ADE (0.4s)	ADE (0.8s)	FDE (0.4s)	FDE (0.8s)
Proposed (Kalman + RPM)	Event	15.35	34.85	31.16	77.76
Kalman Filter (Vanilla)	Event	16.40	37.08	32.80	81.54
Linear Extrapolation	Event	16.45	37.93	32.90	83.32
CNN + Transformer	RGB + Event	121.3	281.5	45.23	85.78
LSTM	None (GT only)	171.9	353.3	53.04	96.44

Observations:

• The proposed method achieved the lowest error across all metrics.
• Even simple Linear Extrapolation outperformed the best Deep Learning models (CNN+Transformer), suggesting DL models suffer from overfitting or poor generalization on this dataset.
• Modulating the Kalman Filter with RPM provided a significant boost (e.g., 4.2px improvement in FDE at 0.8s) over the vanilla Kalman Filter.
• The method performed consistently well across night, rain, and indoor scenarios.

5. Significance and Conclusion

This work demonstrates that classic algorithms combined with event camera physics can outperform complex Deep Learning models for drone trajectory forecasting.

• Efficiency: The method requires no training data, minimal parameter tuning, and operates in real-time.
• Reliability: It provides robust performance in high-speed and low-light conditions where RGB sensors fail.
• Implication: For non-cooperative drone tracking, extracting physical properties (like RPM) from event streams offers a more reliable and generalizable path to accurate state estimation than purely data-driven approaches.

Future Work: The authors plan to estimate individual propeller RPMs (rather than a single drone average) for finer control during erratic movements and to estimate drone orientation based on propeller detection.