🔬 optics

Single Pixel Image Classification using an Ultrafast Digital Light Projector

This paper demonstrates ultrafast, multi-kHz image classification of MNIST digits by combining a microLED-on-CMOS digital light projector for single-pixel imaging with low-complexity machine learning models, achieving real-time inference through a spatiotemporal transformation that bypasses the need for image reconstruction.

Original authors: Aisha Kanwal, Graeme E. Johnstone, Fahimeh Dehkhoda, Johannes H. Herrnsdorf, Robert K. Henderson, Martin D. Dawson, Xavier Porte, Michael J. Strain

Published 2026-03-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Aisha Kanwal, Graeme E. Johnstone, Fahimeh Dehkhoda, Johannes H. Herrnsdorf, Robert K. Henderson, Martin D. Dawson, Xavier Porte, Michael J. Strain

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to recognize a friend in a dark room, but you can't turn on the lights. Instead, you have a magical flashlight that can project thousands of different shadow patterns onto your friend's face. You have a single light sensor on the wall that just tells you "how much light is bouncing back" for each pattern.

That is essentially what this paper is about. The researchers built a super-fast system that can recognize handwritten numbers (like the digits 0–9) without ever actually taking a "picture" in the traditional sense.

Here is a breakdown of their invention using simple analogies:

1. The Problem: The Slow Camera vs. The Fast Eye

Traditional cameras are like a grid of millions of tiny buckets (pixels). To take a photo, every single bucket has to fill up with light, and then the computer has to read all of them. This takes time. If you want to see something moving incredibly fast (like a bullet or a speeding car), a normal camera might be too slow or the data it produces is too heavy to process instantly.

The researchers wanted a way to "see" fast without needing a million buckets.

2. The Solution: The "Single Pixel" Detective

Instead of a million buckets, they used one very fast light sensor (a single pixel). But a single sensor can't see an image on its own. So, they used a trick called Single Pixel Imaging (SPI).

The Analogy: Imagine you are trying to guess what a hidden object looks like. You shine a flashlight through a stencil with a specific shape (like a star) onto the object. Your single sensor tells you, "I see a lot of light!" (meaning the object has a lot of star-shaped parts). Then you switch the stencil to a circle, then a square, then a zigzag.
By flashing thousands of these patterns very quickly and recording how much light bounces back each time, a computer can mathematically figure out what the object looks like.

3. The Super-Speed Upgrade: The MicroLED Flashlight

Usually, these stencils are made by a mechanical device (like a DMD) that flips tiny mirrors. It's fast, but not fast enough for ultra-high-speed tasks.

The researchers used a microLED projector (a screen made of tiny, super-fast LEDs).

The Metaphor: If a traditional projector is like a flip-book where you have to manually turn the pages, this microLED projector is like a high-speed video screen that can change the image 330,000 times per second.
This allowed them to flash their patterns so fast that they could process images at 1,200 frames per second (kfps). That is thousands of times faster than a standard movie camera.

4. The Brain: Skipping the "Reconstruction" Step

Usually, after collecting all that light data, you have to do heavy math to "reconstruct" the full image (put the puzzle back together) and then look at it to guess what it is. This takes time and computing power.

The researchers asked: "Do we really need to see the picture to know what it is?"

The Analogy: Imagine you are a chef tasting a soup. You don't need to see the whole pot of soup to know if it's salty; you just need to taste a spoonful.
They trained two types of "AI brains" (Machine Learning models) to look at the raw stream of light data (the spoonful) and guess the number immediately, without ever rebuilding the full image.
- Model A (ELM): A simple, fast brain that makes quick guesses. It's like a seasoned chef who can tell you "This is a 4" instantly.
- Model B (DNN): A deeper, more complex brain that learns patterns over time. It's like a master chef who analyzes the flavor profile in depth.

5. The Results: Speed vs. Accuracy

They tested this on the famous "MNIST" dataset (handwritten numbers).

The Findings: Even though they didn't reconstruct the full image, their system got over 90% accuracy at incredibly high speeds.
The "Secret Sauce": They discovered that not all patterns are created equal. Some patterns (the ones with simple, smooth lines) contain the most important information for recognizing a number. The complex, noisy patterns were less useful. By using only the "best" patterns, they could make the system even faster without losing much accuracy.

6. Why Does This Matter?

This technology is a game-changer for Machine Vision (giving computers eyes).

Self-Driving Cars: Imagine a car that needs to recognize a pedestrian or a sign in a split second. This system could process that information faster than a human eye can blink.
Anomaly Detection: The system is so good at spotting "what doesn't belong" (like a 4 looking like a 9) that it could be used to instantly spot defects in a factory line or dangerous objects in a security scan, even if the object is moving at high speed.

Summary

The researchers built a super-fast, single-sensor camera that uses a high-speed LED projector to flash patterns at objects. Instead of taking a photo and then analyzing it, they trained an AI to guess the object directly from the light data. It's like recognizing a song by hearing just a few notes, rather than waiting for the whole song to finish. This opens the door for machines to "see" and react to the world at speeds previously thought impossible.

1. Problem Statement

Traditional machine vision systems relying on conventional CMOS cameras face bandwidth limitations, particularly in high-speed applications (e.g., autonomous vehicles) and at unconventional wavelengths where silicon sensors are ineffective. While event-based cameras offer high-speed alternatives, they are restricted to specific wavelength ranges.
Single Pixel Imaging (SPI) offers a solution by using a single-point detector and structured illumination to capture images, enabling operation at high speeds and non-standard wavelengths. However, traditional SPI systems using Digital Micromirror Devices (DMDs) are limited by mechanical switching speeds (~10 kHz), resulting in image generation rates comparable to standard cameras. Furthermore, standard SPI workflows involve reconstructing the full image before classification, which adds computational overhead and latency.
The Core Challenge: How to achieve ultrafast image classification (multi-kHz frame rates) without the latency of image reconstruction, while maintaining high accuracy using compressed sensing and low-complexity machine learning models.

2. Methodology

A. Hardware Setup: Ultrafast SPI

The authors constructed an experimental Single Pixel Image Classification (SPIC) system utilizing a microLED-on-CMOS digital light projector instead of a DMD.

Light Source: A microLED array (128 × 128 pixels) capable of switching patterns at 330,000 fps (global shutter), significantly faster than mechanical DMDs.
Pattern Generation: The system projects Hadamard patterns (binary orthogonal patterns) onto the target object (handwritten digits from the MNIST dataset displayed on a DMD).
Detection: A fast photomultiplier (Onsemi Silicon Photomultiplier) captures the superposition of light reflected from the object for each pattern.
Signal Acquisition: The photodetector output is recorded as a time-series signal by a 1 GHz bandwidth real-time oscilloscope.
Key Innovation: The system bypasses image reconstruction entirely. The raw time-series data (representing the correlation between patterns and the object) is fed directly into a machine learning model.

B. Machine Learning Models

Two low-complexity models were trained to classify the raw time-series data directly:

Extreme Learning Machine (ELM):
- A single-hidden-layer neural network where input weights are randomly initialized and fixed.
- Only output weights are trained using Ridge Regression (closed-form solution), making training extremely fast and avoiding local minima.
- Used for both multiclass classification (0–9) and binary "one-vs-all" classification (anomaly detection).
Deep Neural Network (DNN):
- A feed-forward network with multiple hidden layers trained via backpropagation using the Adam optimizer.
- Used as a benchmark to evaluate the potential of nonlinear feature extraction compared to the simpler ELM.

C. Experimental Protocol

Dataset: MNIST (70,000 handwritten digits). Images were binarized and scaled to fit the DMD.
Pattern Subsets: The study investigated the trade-off between bandwidth and accuracy by using subsets of the full 144-pattern Hadamard set (Had12).
Noise Analysis: Gaussian noise was injected into the input layer to simulate signal degradation and test robustness.

3. Key Contributions

Ultrafast Classification without Reconstruction: Demonstrated image classification at 1.2 kfps (1,200 frames per second) by combining microLED-based SPI with direct spatiotemporal classification, completely bypassing the computationally expensive image reconstruction step.
Hardware Advancement: Validated the use of microLED-on-CMOS projectors for SPI, achieving pattern generation speeds ~100 times faster than traditional DMDs.
Efficient Anomaly Detection: Showed that a minimalist ELM model can achieve >99% accuracy in binary (one-vs-all) classification, making it highly suitable for real-time anomaly detection in high-speed scenarios.
Pattern Selection Strategy: Identified that the spatial frequency content of Hadamard patterns dictates classification efficiency. Patterns with fewer inversions (lower spatial frequency, "Cat1") contain the most critical information for classification, allowing for significant bandwidth reduction without severe accuracy loss.
Noise vs. Information Loss: Distinguished that performance degradation in compressed sensing is primarily due to the loss of structured spatial information rather than simple signal-to-noise ratio (SNR) reduction.

4. Results

Classification Accuracy:
- ELM Model: Achieved >90% accuracy for multiclass classification and >99% for binary classification using the full Had12 pattern set.
- DNN Model: Achieved >90% accuracy with the full pattern set, closely matching the performance of numerically simulated binarized MNIST.
Bandwidth vs. Accuracy Trade-off:
- Reducing the number of patterns increases the effective bandwidth.
- Using only the first 1/4 of the Had12 patterns (lowest spatial frequency) maintained ~78% accuracy while drastically increasing the frame rate.
- Using the last 1/4 of patterns (highest spatial frequency) resulted in significantly lower accuracy, confirming that low-frequency patterns are more informative for digit recognition.
Inference Speed:
- ELM: 31 $\mu$ s per digit (2x faster than DNN).
- DNN: 73 $\mu$ s per digit.
- Both models allow for real-time processing where the inference time is comparable to the image generation time.
Robustness: The system remained robust against additive Gaussian noise (up to $\sigma=0.5$ ), maintaining >95% accuracy. Performance only dropped significantly at very high noise levels ( $\sigma=1.0$ ).

5. Significance and Impact

This work represents a significant step forward in optical machine vision and edge computing:

Real-Time Capability: By eliminating image reconstruction and utilizing ultrafast microLEDs, the system operates at speeds previously unattainable for SPI, making it viable for dynamic environments like autonomous driving.
Hardware Efficiency: The approach reduces hardware complexity on the detection side (single pixel vs. megapixel array) and enables operation at wavelengths where standard cameras fail.
Algorithmic Efficiency: The success of the ELM model demonstrates that complex deep learning is not always necessary for high-speed SPI tasks; simple, fast models can achieve near-optimal results when paired with the right optical encoding.
Anomaly Detection: The high accuracy of the binary classifier suggests immediate applications in industrial quality control and safety systems where detecting "outliers" in a high-speed stream is critical.

In summary, the paper successfully bridges the gap between ultrafast optical hardware and efficient machine learning, proving that reconstruction-free single pixel imaging is a viable, high-performance solution for next-generation machine vision.