🔬 optics

Learning Binary Sampling Patterns for Single-Pixel Imaging using Bilevel Optimisation

This paper proposes a bilevel optimization method using a Straight-Through Estimator and learned variational regularization to design task-specific binary illumination patterns for single-pixel imaging, demonstrating superior reconstruction performance in highly undersampled and data-scarce scenarios.

Original authors: Serban Cristian Tudosie, Alexander Denker, Zeljko Kereta, Simon Arridge

Published 2026-04-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Serban Cristian Tudosie, Alexander Denker, Zeljko Kereta, Simon Arridge

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to take a high-quality photo of a beautiful, intricate piece of jewelry, but there is a catch: you don't have a camera. Instead, you only have a single, tiny light sensor (like a single pixel) and a flashlight that can project different patterns of light onto the jewelry.

To "see" the object, you have to shine a series of different light patterns on it, record how much light bounces back for each pattern, and then use math to piece the image back together. This is called Single-Pixel Imaging (SPI).

The Problem: The "Flashlight Puzzle"

The big challenge is: What patterns should you shine?

If you use random patterns, the image might look blurry or "blocky."
If you use standard patterns (like a checkerboard), you might miss the fine details.
If you try to use too many patterns, it takes forever to take the "photo."

In the past, scientists used "handcrafted" patterns—basically, patterns humans thought looked good. But humans aren't always the best at solving complex mathematical puzzles.

The Solution: The "Smart Flashlight"

The researchers in this paper decided to stop guessing. Instead of using human-designed patterns, they taught a computer to learn the perfect patterns specifically for the object it is looking at.

Think of it like this: Imagine you are trying to find a hidden object in a dark room using only a flashlight.

Old Method: You walk around shining the light in a standard grid pattern. It works, but it's slow and you might miss the object in a corner.
This Paper's Method: You have a "smart flashlight" that learns from experience. It realizes, "Hey, every time I shine the light in this specific star shape, I get a much better hint about where the object is!"

How They Did It (The "Secret Sauce")

There were two big technical hurdles they had to jump over, which they solved with clever "tricks":

The "On/Off" Problem (Bilevel Optimization & STE):
Most computer learning works by making tiny, smooth adjustments (like turning a dimmer switch). But real-world SPI hardware is "binary"—the light is either ON or OFF. You can't have a "half-on" light. This makes the math very "jumpy" and difficult for computers to learn.
- The Analogy: It’s like trying to teach someone to walk, but they can only take steps that are exactly 1 foot long—no smaller, no larger. You can't "nudge" them.
- The Fix: They used a trick called the Straight-Through Estimator (STE). It’s like telling the computer, "Pretend you can use a dimmer switch while you're practicing, but when you actually perform, you have to flip the hard ON/OFF switch." This allows the computer to learn smoothly while still producing a practical, binary result.
The "Expert Eye" (Learned Regularization):
When the computer tries to reconstruct the image from the light measurements, it needs an "eye" to help it fill in the gaps. Instead of using a generic "eye," they used a highly trained, "expert eye" (called a regularizer) that already knows what real biological cells look like. This helps the computer distinguish between actual detail and random noise.

Why This Matters

The results were impressive. The researchers found that:

Speed: Their "smart patterns" could produce a clear image using much less light than traditional methods. It’s like getting a high-def photo using only 1/4 of the usual flashes.
Efficiency: Even if the computer only saw a tiny bit of data (a "scarce-data" setting), it could still figure out the image.
Accuracy: The images were much sharper and more detailed, especially when looking at tiny things like cells in a microscope.

In short: They turned a blind, guessing process into a smart, learning process, allowing us to take high-quality pictures with much simpler (and cheaper) equipment.

Technical Summary: Learning Binary Sampling Patterns for Single-Pixel Imaging using Bilevel Optimisation

1. Problem Statement

Single-Pixel Imaging (SPI) is a cost-effective imaging technique that reconstructs objects using a single photodetector by sequentially illuminating the target with structured light patterns. A critical challenge in SPI is the design of these illumination patterns, especially in highly undersampled regimes (where the number of measurements $M$ is much smaller than the number of pixels $N$ ).

The authors identify three primary constraints in existing literature:

Hardware Constraints: Practical SPI hardware (like Digital Micromirror Devices) only supports unipolar binary patterns $\{0, 1\}$ , though mathematical models often use bipolar $\{-1, 1\}$ patterns.
Optimization Complexity: Optimizing binary patterns is a non-convex, non-differentiable, and combinatorial problem ( $2^{MN}$ search space), making standard gradient descent impossible.
Data Efficiency: Existing deep learning methods (like DCAN) require massive datasets (tens of thousands of images) because they jointly optimize the patterns and the reconstruction network.

2. Methodology

The paper proposes a bilevel optimization framework that decouples pattern learning from reconstruction training.

A. Bilevel Formulation:
The problem is structured into two levels:

Upper-level: Find the optimal latent continuous matrix $Z$ (where the binary pattern $A = \text{sgn}(Z)$ ) and the regularization weight $\alpha$ to minimize the reconstruction error over a dataset.
Lower-level: Solve the variational reconstruction problem for a given $A$ and $\alpha$ :
$\hat{x} \in \arg\min_x \frac{1}{2}\|Ax - y\|_2^2 + \alpha J(x)$
where $J(x)$ is a pre-trained learned regularizer (e.g., Total Deep Variation or Weakly Convex Ridge Regularizer).

B. Key Algorithmic Innovations:

Straight-Through Estimator (STE): To handle the non-differentiable $\text{sgn}(Z)$ function, the authors use STE. During the backward pass, they replace the zero gradient of the sign function with a surrogate gradient (e.g., Identity, Clipped, or Tanh). The paper provides a probabilistic interpretation, showing that these surrogates correspond to convolving the sign function with specific noise distributions (Uniform or Logistic).
Jacobian-Free Backpropagation (JFB): To avoid the massive computational cost of differentiating through the lower-level solver's iterations, they use a zero-order Neumann series approximation to compute the gradient efficiently.
Decoupling: By using fixed, pre-trained regularizers instead of training a neural network from scratch, the method significantly reduces the amount of data needed for training.

3. Key Contributions

Bilevel Framework for Binary Patterns: A robust method to learn task-specific binary patterns directly from data.
STE Implementation: A mathematically grounded approach to optimizing discrete binary matrices using continuous latent variables.
Data Efficiency: Demonstrates that high-quality patterns can be learned with as few as 250 images, compared to the $\sim$ 30,000 required by end-to-end deep learning.
Theoretical Insight: Provides a probabilistic link between common STE surrogate gradients and noise-smoothed versions of the sign function.

4. Results

The method was evaluated on the CytoImageNet microscopy dataset.

Superior Reconstruction: The learned patterns (STE-TDV/WCRR) significantly outperform classical patterns (Gaussian and Scrambled Hadamard) in terms of PSNR and SSIM, particularly in extreme undersampling (e.g., $M=128$ or $256$).
Efficiency Gain: The learned patterns can achieve the same reconstruction quality as Scrambled Hadamard patterns while using four times fewer measurements.
Robustness: The STE approach proved more robust to noise mismatches and provided more stable training than the "Relax-and-Penalise" (RnP) approach.
Global Optimality: Small-scale experiments (3x3 images) confirmed that the STE solution closely aligns with the global optimum or reaches a near-optimal local minimum.

5. Significance

This work shifts the paradigm of SPI pattern design from "handcrafted/fixed" to "data-driven/learned." By decoupling the pattern optimization from the reconstruction architecture, the authors have made deep-learning-based SPI accessible for biomedical applications where large-scale datasets are often unavailable. The ability to maintain high image fidelity with minimal measurements has direct implications for high-speed, low-dose fluorescence microscopy and infrared imaging.