💻 computer science

Extending Deep Event Visual Odometry with Sparse Point-Cloud Export

This paper extends the Deep Event Visual Odometry (DEVO) framework by adding a sparse point-cloud export pipeline that converts the system's internal 3D structure into an explicit geometric representation for visualization and further processing, achieving high local precision on benchmark data while preserving the original odometry performance.

Original authors: Alireza Safdari, Sajad Ashraf

Published 2026-05-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Alireza Safdari, Sajad Ashraf

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are driving a car at night through a heavy storm. A regular camera (like the one in your phone) would be useless; the rain would blur the image, and the darkness would make it impossible to see. But imagine if your car had a special "event camera" instead. This camera doesn't take full pictures. Instead, it acts like a hyper-sensitive alarm system: it only "screams" (sends a signal) when something changes—like a tree branch moving or a streetlight flickering. It ignores the static, dark parts of the scene and focuses entirely on the motion and changes.

The Original System (DEVO)
The paper starts with a system called DEVO (Deep Event Visual Odometry). Think of DEVO as a very smart driver who only looks at the "screaming" parts of the event camera.

How it works: It picks out a few interesting spots (like the edge of a sign or a moving car), tracks how they move, and calculates exactly where the car is going.
The Limitation: DEVO is excellent at telling you where you are and where you are going (the trajectory). However, it keeps the map of the world inside its own brain. It knows the 3D shape of the objects it tracked, but it doesn't "show" you the map. It's like a GPS that knows the road but refuses to print you a map.

The New Addition (Sparse Point-Cloud Export)
This project is like adding a "Map Printer" to that smart driver. The authors didn't change how the driver drives or how it calculates the route. Instead, they built a side-door that lets you peek into the driver's brain and grab the 3D map it was already building.

The "Sparse" Concept: Because the event camera only sees changes, the map it builds isn't a solid, solid wall of pixels like a photograph. It's more like a constellation of stars. You have thousands of individual points (stars) that represent the edges and corners of objects, but the spaces in between are empty.
The Process: The new system takes those internal "stars" (the tracked patches and depth estimates), converts them into a standard 3D format (a point cloud), and saves them so you can look at them, clean them up, or use them for other tasks.

The Experiment: The "Board" Test
To test this, the researchers used a sequence called "BOARD SLOW." Imagine a large board with geometric shapes (diamonds, squares, circles) printed on it.

Why this board? Event cameras love edges. When the camera moves, the edges of the shapes create the most "events" (alarms). The flat, empty spaces inside the shapes create nothing.
The Result: The system successfully exported a 3D cloud of points. When they looked at the result, the points lined up perfectly with the edges of the shapes on the board. It was like the system had drawn the outlines of the shapes in 3D space using invisible ink.

What They Found (The Good and The Bad)

It Works Locally: The points the system exported were very accurate. If you looked at a specific spot, the points were right where they should be (98% precision).
It's Not a Solid Wall: Because the system only tracks a few "interesting" spots, the final map is sparse. It's not a complete, solid 3D model of the room. It's a collection of dots.
The "Density" Problem: The researchers compared their system to a "Gold Standard" method (EMVS) that uses a perfect, known path. They found that the Gold Standard made a much fuller map. Why? Because the Gold Standard had 4 times more "snapshots" (poses) to work with.
- Analogy: Imagine trying to draw a picture of a building. The Gold Standard took 4,000 photos from different angles. DEVO only took 1,000. Both drew the same building, but the Gold Standard's drawing was much more detailed because it had more data points. The difference wasn't because DEVO was "wrong" about the size; it was just that DEVO didn't look at the building from as many angles.

The Conclusion
The paper concludes that they successfully turned a "GPS-only" system into a "GPS + Map Printer."

What it is: A tool that takes the hidden 3D data from a high-speed, event-based camera and turns it into a usable list of 3D points.
What it isn't: It is not a magic wand that creates a perfect, solid 3D model of a room. It is a "sparse" model, meaning it's a collection of dots that outline the important parts of the scene.
Why it matters: For robots or drones moving fast in the dark, having a rough 3D sketch of the environment (the edges of walls, the corners of obstacles) is incredibly useful, even if it's not a perfect photograph. This project gives engineers a way to get that sketch directly from the event camera's brain.

Technical Summary: Extending Deep Event Visual Odometry with Sparse Point-Cloud Export

Problem Statement
Deep Event Visual Odometry (DEVO) has established that monocular event-only odometry can achieve robust performance under high-speed motion and challenging lighting by combining sparse patch tracking, learned patch selection, recurrent correspondence refinement, and differentiable bundle adjustment. However, DEVO is primarily designed as a trajectory estimation system. While it internally maintains sparse geometric structure (tracked patch locations and estimated depths) to optimize camera poses, this 3D information remains latent within the optimization loop and is not exposed as an explicit output. For many robotics, debugging, and visualization tasks, trajectory estimation alone is insufficient; an exportable 3D point cloud is required for scene inspection and downstream processing. Existing event-based 3D reconstruction methods (e.g., EMVS) treat reconstruction as a separate problem, often requiring dedicated pipelines or additional sensor modalities.

Methodology
The proposed work extends DEVO by adding a sparse point-cloud export pipeline without altering the core visual odometry formulation. The approach operates on the observation that DEVO already estimates the necessary ingredients for 3D reconstruction: camera poses and sparse depth variables associated with tracked patches.

Internal Exposure: The core modification involves exposing the internal observables of the DEVO inference pipeline. The system was extended with an opt-in flag (return_observables=True) in helper functions (run_rgb, run_voxel_norm_seq, run_voxel). When enabled, these functions return not only camera poses and timestamps but also the current sparse point cloud and depth estimates.
3D Generation: The export process utilizes standard back-projection. For each valid tracked patch with image coordinates $(u, v)$ and estimated depth $z$ , the 3D point in camera coordinates is calculated using the intrinsic matrix $K$ ( $p_c = zK^{-1}[u, v, 1]^T$ ). These points are then transformed into a common world coordinate frame using the optimized camera poses.
Pipeline and Post-Processing: A practical workflow was implemented to handle data extraction, format conversion (to .ply, .npy), and cleanup. The pipeline includes optional post-processing steps such as statistical and radius outlier removal to improve the usability of the sparse, noisy output without modifying the underlying odometry estimates.
Implementation: The extension preserves the original DEVO codebase structure, adding a dedicated export script (scripts/export_pointcloud.py) and an orchestration script (scripts/start_pipline.py). A Docker container was provided to ensure reproducibility across different environments.

Key Contributions

Sparse Representation Exposure: The project extends DEVO to expose its internal sparse 3D representation as exportable point-cloud data, transforming a trajectory-only system into one that provides geometric scene output.
Practical Export Pipeline: Implementation of a complete workflow for point-cloud generation, format conversion, and cleanup, including support for various dataset types (FPV, RPG, VECtor).
Analysis of Limitations: An empirical analysis of the usefulness and limitations of extracting sparse point clouds from an event-based visual odometry framework, specifically regarding density, completeness, and sensitivity to accumulated odometry noise.

Experimental Results
The system was evaluated on the BOARD SLOW event-camera sequence, which features a planar calibration board with high-contrast geometric patterns. The exported DEVO SLAM point cloud was compared against EMVS reconstructions generated using both DEVO poses and ground-truth poses.

Scale Isolation: Experiments confirmed that the difference in reconstruction quality between ground-truth-pose EMVS and DEVO-pose EMVS is not caused by the global scale mismatch (DEVO trajectory was ~1.42× larger). Scaling the trajectory and depth range consistently did not improve the DEVO-pose reconstruction quality.
Pose Density: The primary factor influencing reconstruction quality was pose density. The ground-truth trajectory provided 4,171 poses compared to DEVO's 985 poses (a ~4.2× difference). This denser sampling provided EMVS with more independent viewpoints, resulting in sharper reconstructions.
Geometric Consistency: The exported DEVO SLAM cloud showed high local consistency with EMVS reconstructions. At a 5 cm threshold, the F-score was 0.491 (after ICP alignment), with a precision of 98.4%. This indicates that almost all EMVS points lie close to the exported DEVO cloud. However, recall was lower (31.7%) due to differences in coverage properties; DEVO accumulates sparse points over the full trajectory, while EMVS reconstructs localized semi-dense structures.
Planarity and Noise: While both methods recovered the dominant planar board geometry, the DEVO SLAM cloud exhibited higher plane-fitting residuals (93.2 mm vs. 13.5 mm for EMVS) and contained more isolated outliers. This is attributed to the accumulation of sparse odometry state and pose drift over the full sequence.
Visual Validation: Projected 3D points aligned with the visible boundaries of the printed shapes (diamonds, squares, circles) on the board, confirming that the exported points correspond to real scene geometry triggered by brightness changes.

Significance and Claims
The paper claims that the proposed extension successfully bridges the gap between pure visual odometry and full 3D reconstruction by exposing latent geometric information within an existing, high-performance odometry framework. The system preserves the original strengths of DEVO (robustness in high-speed and low-light conditions) while adding utility for scene inspection and downstream processing.

The authors modestly characterize the output not as a dense surface reconstruction comparable to dedicated multi-view stereo methods like EMVS, but as a sparse structural representation. The significance lies in demonstrating that event-based visual odometry pipelines can be extended to provide usable 3D scene geometry without replacing the core optimization logic. The work highlights that while the resulting point cloud is inherently sparse and sensitive to odometry noise, it is geometrically valid locally and serves as a valuable intermediate step for robotics workflows requiring lightweight geometric representations. Future work is suggested to focus on denser temporal fusion and tighter integration with dedicated reconstruction methods to improve completeness.

Technical Summary: Extending Deep Event Visual Odometry with Sparse Point-Cloud Export

More like this