🤖 machine learning

Generative Anonymization in Event Streams

This paper introduces the first generative anonymization framework for neuromorphic event streams that resolves the utility-privacy trade-off by synthesizing realistic but non-existent identities to prevent human identification while preserving the structural integrity required for downstream vision tasks, accompanied by a novel synchronized real-world dataset for rigorous evaluation.

Original authors: Adam T. Müller, Mihai Kocsis, Nicolaj C. Stache

Published 2026-04-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Adam T. Müller, Mihai Kocsis, Nicolaj C. Stache

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-powered security camera that doesn't take normal photos. Instead of capturing a steady stream of images like a movie, it only records changes. If a leaf doesn't move, the camera sees nothing. If a person waves, the camera sees a spark. This is called an Event Camera.

These cameras are amazing: they are incredibly fast, work in total darkness, and use very little battery. They are perfect for self-driving cars and robots.

The Problem: The "Ghost" in the Machine
For a long time, scientists thought these cameras were safe for privacy. They thought, "Since it only sees movement and not clear pictures, it can't show who you are."

But recently, AI got so smart that it could take these sparse "sparks" of movement and reconstruct a clear, high-definition video of a person's face. Suddenly, a privacy-safe camera became a privacy nightmare. If you wanted to use these cameras in a public park, you'd have to choose: either protect people's faces (and ruin the data so robots can't see anything) or keep the data useful (and risk exposing everyone's identity).

The Solution: The "Digital Mask" That Moves
This paper introduces a clever new way to solve this. Instead of blurring the face or scrambling the data (which makes the robot blind), they use Generative AI to swap the person's face with a completely fake, non-existent person.

Think of it like this:

Old Way (Blurring): Imagine trying to hide a face by smearing paint over it. The robot can no longer tell where the head is, or if the person is smiling. The data is broken.
New Way (Generative Swap): Imagine a skilled actor stepping onto a stage. The actor wears a mask that looks like a different person, but they mimic the original person's movements perfectly. They turn their head, smile, and frown exactly as the original person did. To the robot, the scene looks perfect and useful. To a spy trying to identify the person, the face is now a stranger.

How They Did It (The "Translator" Pipeline)
The tricky part is that Event Cameras speak a different language (sparks) than the AI models that swap faces (which speak normal video).

Translation: First, they translate the "sparks" into a normal, black-and-white video so the AI can understand it.
The Swap: They use a powerful AI to find the face and swap it with a fake one, keeping all the expressions and head movements intact.
Back-Translation: They translate this new, fake video back into "sparks" for the Event Camera.
Seamless Stitching: They carefully cut out the old face from the original "spark" stream and paste in the new "spark" stream of the fake face, making sure the edges blend smoothly so the robot doesn't see a glitch.

The Result: A New Dataset
To prove this works, the authors built a special robot arm that moved a camera around a person in a controlled way. This created a brand-new dataset where they could test their method.

Their tests showed:

Privacy: If you try to reconstruct the video from the "sparks," you see the fake face, not the real person. The real identity is gone.
Utility: The robot can still see the person, track their head, and recognize that they are smiling. The data is still useful for safety and navigation.

Why This Matters
This is the first time anyone has successfully used "generative" (creative) AI to protect privacy in this specific type of camera. It means we can finally put these super-fast, low-power cameras in schools, hospitals, and streets without worrying about violating people's privacy. It's like giving the camera a pair of glasses that lets it see what is happening, but forgets who is doing it.

1. Problem Statement

Neuromorphic vision sensors (event cameras) offer significant advantages over traditional cameras, including microsecond latency, high dynamic range, and low power consumption. However, their deployment in public spaces raises critical privacy concerns.

The Privacy Vulnerability: While event streams inherently lack absolute intensity information, recent Event-to-Video (E2V) deep learning models can reconstruct high-fidelity intensity images from sparse event streams. This capability inadvertently exposes human biometric identifiers (faces), allowing for identity recovery.
The Utility-Privacy Trade-off: Existing anonymization methods for event streams (e.g., spatial scrambling, adversarial noise injection) are "destructive." They corrupt the spatio-temporal structure of the data to hide identities, which severely degrades the data's utility for downstream perception tasks like pose estimation or tracking.
The Gap: There is a lack of methods that can anonymize event data while preserving the structural integrity required for machine vision tasks. Furthermore, there is a scarcity of high-quality, real-world synchronized event-RGB datasets for rigorous evaluation.

2. Methodology

The authors propose the first generative anonymization framework for event streams. Instead of destroying data, the method synthesizes realistic, non-existent identities to replace the original subject while preserving the underlying spatio-temporal structure. The pipeline bridges the modality gap between asynchronous events and standard spatial generative models through an intermediate intensity representation.

Key Steps in the Pipeline:

Event-to-Frame Projection: Raw asynchronous event streams ( $E$ ) are converted into a set of synchronized grayscale intensity frames using the EVREAL evaluation framework.
Generative Anonymization (Face Swapping):
- Faces are detected in the intermediate frames.
- A synthetic, non-existent identity is generated using Stable Diffusion 2 (SD2).
- The INSwapper model performs face swapping, replacing the original subject with the synthetic identity while retaining semantic attributes (gaze, pose).
- The output is upsampled (FSRCNN) and enhanced (CLAHE) to improve resolution.
Temporal Interpolation & Spatial Feathering:
- Interpolation: Since event cameras have microsecond resolution, discrete bounding boxes from the frames are interpolated linearly to create a continuous 3D bounding box function $B(t)$ for any event timestamp.
- Feathering: To avoid harsh artificial boundaries when merging the new face with the background, a stochastic spatial feathering mechanism is applied. Events near the bounding box edge are retained with a probability decaying via a half-Gaussian distribution, creating a smooth gradient.
Frame-to-Event Re-encoding (V2E): The anonymized grayscale frames are projected back into the neuromorphic domain using the v2e simulator.
Stream Compositing: The synthesized facial events are spatially warped and temporally aligned with the original background event stream (excluding the original face) to create the final anonymized event stream ( $E_{final}$ ).

3. Key Contributions

Generative Anonymization Framework: The first pipeline to apply generative identity replacement to event streams, successfully resolving the utility-privacy trade-off by preserving spatio-temporal structure.
Novel Dataset: Introduction of a synchronized real-world Event-RGB dataset captured using a collaborative robot (cobot). This setup ensures precise, reproducible camera motion (egomotion) while the subject remains static (reading text), providing a robust benchmark for evaluating spatiotemporal integrity.
Evaluation Metrics: Proposes specific metrics for the event domain:
- Spatiotemporal Chamfer Distance (STCD): Measures structural preservation of the event point cloud.
- Event Mover's Distance (EMD): Approximated via Sliced Wasserstein Distance to measure global distributional shifts in event density.
Comprehensive Evaluation: Demonstrates that the method prevents identity recovery while maintaining high utility for downstream tasks.

4. Experimental Results

The framework was evaluated using the FireNet architecture for E2V reconstruction and YOLOv8/Event-based detectors for downstream tasks.

Anonymization Effectiveness:
- Identity Similarity: Dropped significantly from a baseline of 0.713 to 0.118, indicating successful obfuscation.
- Structural Shift: The STCD increased by over 31x compared to the baseline, and EMD increased from 0.0085 to 0.1276, confirming a distinct structural change in the event space.
Feature Preservation (Utility):
- Temporal Stability: Remained consistent (0.760 $\to$ 0.770), showing the synthetic identity is coherent over time.
- Pose & Mimicry: The method achieved a Pose Error of 3.304° and a Mimicry Error of 0.181 (tighter than the natural intra-subject variance of 0.239), proving that facial expressions and head movements are faithfully preserved.
Downstream Performance:
- Detection: Face detection rates remained unchanged (0% error).
- Alignment: Bounding box alignment was high, with a YOLO IoU of 0.960 (grayscale) and Event IoU of 0.702.
- Conclusion: The anonymized data is fully usable for standard computer vision tasks.

5. Limitations

Modality Gap: The method relies on an intermediate frame-based step. It does not yet perform native, asynchronous generative modeling directly on event spikes.
V2E Discretization: The reverse projection (V2E) introduces discretization artifacts. In high-frequency regions (like the mouth), this can lead to "smearing" or insufficient event density, which may affect downstream E2V reconstruction quality.
Resolution Constraints: The fidelity of micro-expressions is limited by the resolution and capabilities of the current face-swapping prior (INSwapper/SD2).

6. Significance

This work represents a paradigm shift in neuromorphic vision privacy. By moving from destructive obfuscation (which ruins data utility) to generative anonymization (which preserves utility), the authors enable the safe deployment of event cameras in privacy-sensitive environments (e.g., autonomous driving, smart surveillance). The introduction of a high-quality, synchronized dataset and novel evaluation metrics sets a new standard for future research in privacy-preserving neuromorphic vision.