💻 computer science

NightSight: Passive Computation for Navigation in Dark Using Events

This paper presents NightSight, a lightweight, real-time perception system for small aerial robots that enables autonomous navigation in complete darkness by combining a monocular event camera, a coded aperture lens, and an infrared dot projector to generate depth-dependent blur signatures, which a neural network trained solely on synthetic data decodes into accurate dense depth maps with zero-shot generalization to complex real-world scenes.

Original authors: Deepak Singh, Brijan Vaghasiya, Shreyas Khobragade, Nitin Sanket

Published 2026-05-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Deepak Singh, Brijan Vaghasiya, Shreyas Khobragade, Nitin Sanket

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a tiny, agile drone trying to navigate a pitch-black cave or a collapsed building after an earthquake. It can't see anything because it's too dark for normal cameras, and it's too small to carry heavy, power-hungry equipment like a giant flashlight or a laser scanner.

This paper presents a clever solution called NightSight. Think of it as giving the drone a pair of "super-eyes" that work in total darkness without needing a heavy battery pack.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Heavy Backpack" Dilemma

Small rescue drones are like nimble acrobats, but they have a weakness: they can't carry heavy gear. Traditional ways to see in the dark involve:

Big Flashlights: They drain the battery fast.
Laser Scanners (LiDAR): They are heavy and bulky.
Standard Cameras: They get "blinded" by darkness or glare.

The authors wanted a way for a tiny drone to "see" depth (how far away things are) using almost no power and very little weight.

2. The Solution: A "Magic Lens" and a "Strobe Light"

The team built a system with three main parts:

An Event Camera: Instead of taking normal photos (like a standard camera that takes a picture 30 times a second), this camera is like a super-fast nervous system. It only "wakes up" and sends a signal when it sees something move or change brightness. It's incredibly sensitive to tiny changes in light.
A Coded Aperture (The Magic Lens): This is a special cover placed over the camera lens. It's not a simple hole; it has a complex pattern cut into it (like a stencil).
A Dot Projector: The drone shines a pattern of invisible infrared dots onto the wall or obstacles in front of it.

3. The Trick: "Blur as a Ruler"

Here is the clever part, which the authors call "Passive Computation."

Imagine you are looking at a sticker on a wall through a special stencil.

If the wall is close, the sticker looks sharp.
If the wall is far away, the sticker doesn't just get blurry; it changes shape. It starts to look exactly like the pattern of the stencil (the "Magic Lens").

In this system, the drone projects dots. As those dots get farther away, the "Magic Lens" makes them stretch and change shape in a very specific way. The shape of the blur tells the computer exactly how far away the object is.

The Analogy: Think of it like a shadow puppet show. If you hold your hand close to the wall, the shadow is sharp. If you move your hand back, the shadow gets fuzzy and takes on the shape of the light source. The drone uses this "fuzziness" to measure distance, but it does it so fast and efficiently that it doesn't need a supercomputer to figure it out.

4. Learning Without Seeing (The "Video Game" Training)

Usually, to teach a computer to recognize depth, you need thousands of real-world photos of dark rooms. But since it's hard to get those, the authors did something smart:

They built a simple robot arm that moved a camera back and forth in front of a flat wall.
They used a computer simulation to teach the AI how the "blur shapes" change with distance.
The Result: The AI learned the rules of the game in a simple, fake environment. When they took the real drone into a complex, dark cave with rocks and pillars, the AI didn't need any new training. It just applied the rules it learned and worked perfectly. This is called "Zero-Shot Generalization."

5. The Results

Speed: The system works in real-time (20 times a second), fast enough for a drone to fly and dodge obstacles.
Accuracy: It can measure distance up to 2.5 meters (about 8 feet) with very high accuracy (only about 7 centimeters of error).
Efficiency: It runs on a small, low-power computer (NVIDIA Jetson Orin Nano) that fits on a tiny drone.

Summary

The paper shows that by combining a special "event" camera, a patterned lens, and a dot projector, a tiny robot can navigate in total darkness. It turns the physics of light and blur into a built-in ruler, allowing the robot to "see" without needing heavy, power-draining equipment. It's like teaching a drone to feel its way through the dark using the shape of shadows rather than just looking at them.

Technical Summary: NightSight – Passive Computation for Navigation in Dark Using Events

Problem Statement
Small aerial robots are ideal for search and rescue in confined, hazardous, and cluttered environments due to their agility and low cost. However, enabling autonomous navigation in complete darkness remains a significant hurdle. Traditional perception systems for low-light environments often rely on heavy LiDARs, high-power illumination (>10W), or thermal cameras, which exceed the payload, power, and computational constraints of small aerial platforms. Furthermore, existing computational approaches using structured illumination and coded optics (e.g., AsterNav) typically employ frame-based cameras, which suffer from limited dynamic range and reduced depth quality in low-light conditions. The core challenge is to achieve robust, real-time depth perception in total darkness (<1 milliLux) using a lightweight, resource-constrained system.

Methodology
The authors propose NightSight, a lightweight perception framework that integrates three key components: a monocular event camera, a coded aperture lens, and an infrared (IR) dot projector. The system operates on the principle of passive computation, where optical components process signals in the physical domain without active electrical power, reducing the onboard computational burden.

Optical Encoding: The system projects a sparse, structured IR dot pattern onto the scene. This pattern is imaged through a custom coded aperture lens. Unlike a standard circular aperture, the coded aperture induces depth-dependent blur signatures. As the scene depth varies, the projected dots appear as scaled versions of the aperture shape. This creates a strong, physics-based cue where the blur pattern size and shape implicitly encode the scene geometry.
Event-Based Sensing: An event camera (silicon retina) captures these patterns. Unlike frame-based cameras, event cameras offer high dynamic range and high temporal resolution, capturing brightness changes asynchronously. The system leverages the fact that sharp regions (in focus) generate more events than blurred regions (out of focus).
Learning-Based Decoding: A Convolutional Neural Network (CNN) is trained to decode these blur signatures into dense metric depth maps.
- Training Regime: The model is trained entirely on synthetic data. The simulation involves projecting the structured pattern onto a simple planar wall. Motion is simulated using a servo-driven linear actuator that oscillates the camera to generate event streams, mimicking camera movement.
- Generalization: The model is never explicitly trained on varying velocities or complex 3D scenes during training, yet it is tested for zero-shot generalization on real-world, complex environments.
Hardware Implementation: The system runs on a NVIDIA Jetson Orin Nano, achieving real-time inference at 20 Hz. The optical setup includes a 16mm f/1.4 lens with 3D-printed coded apertures (various designs tested, including pinhole, fully open, Levin, Zhou1, Zhou2, and a W-shaped aperture inspired by cuttlefish eyes).

Key Contributions

Passive Computation Framework: The paper presents a novel monocular depth sensing approach for aerial robots in complete darkness by combining event cameras, structured illumination, and coded apertures. It shifts part of the perception pipeline from digital processing to passive optics.
Zero-Shot Generalization: The authors demonstrate that a model trained solely on synthetic data generated from a simple planar wall setup can generalize effectively to complex, real-world scenes with varying obstacle sizes, shapes, and textures without retraining.
Aperture Benchmarking: The work benchmarks multiple coded aperture patterns to evaluate their effectiveness for event-based depth estimation, identifying specific designs that optimize depth cues.
Resource Efficiency: The system achieves accurate metric depth estimation while operating on a low-power embedded platform (Jetson Orin Nano), making it suitable for small, payload-constrained robots.

Results

Accuracy: The system achieves high accuracy with an L1 error of 7.0 cm over a depth range of 2.5 meters, corresponding to a 2.80% error rate.
Optimal Configuration: The W-shaped coded aperture (inspired by cuttlefish eyes) yielded the best performance, particularly at depths farther from the focal plane, outperforming pinhole, fully open, and other coded aperture designs.
Focus Distance Sensitivity: Experiments showed that a focus distance ( $Z_f$ ) of 0.50 m provided the best balance, whereas $Z_f = 0.25$ m resulted in higher prediction errors due to excessive sensitivity to small depth variations.
Real-World Validation: In a real-world flight test in complete darkness (<1 milliLux), the system successfully navigated through cylindrical obstacles.
Comparative Performance: When compared to a state-of-the-art monocular depth model (DepthPro) running on a high-end DSLR (Nikon D5600) with extreme exposure settings (1s shutter, ISO 25600), the DSLR failed to perceive the scene structure. Conversely, the NightSight system, even with the event camera's high dynamic range, required the structured IR illumination to recover meaningful scene structure and accurate depth, highlighting the necessity of the active illumination component in total darkness.

Significance and Claims
The paper claims that NightSight highlights the potential of combining structured illumination, coded optics, and event-based sensing to enable robust perception in complete darkness. By adhering to a principle of parsimony—similar to biological sensing strategies used by insects—the system extracts task-relevant information directly through physical interactions (optics) rather than extensive digital computation. The authors assert that this approach solves the fundamental challenge of perception for small aerial robots in dark environments without requiring heavy payloads or high-power illumination, offering a viable path for autonomous navigation in scenarios where traditional sensors fail. The work is modest in its scope, focusing on the feasibility of the approach and its generalization capabilities, while noting that future work will aim to increase the depth sensing range for high-speed navigation in cluttered dark scenes.