Unbiased Gradient Estimation for Event Binning via Functional Backpropagation

1. Problem Statement

Event-based vision sensors capture scenes as asynchronous streams of spikes (events) rather than synchronous frames. To utilize standard deep learning pipelines, these events are typically aggregated into frames via binning functions. However, this process introduces a fundamental challenge:

• Discontinuity: Binning functions are inherently discontinuous (e.g., counting events in a pixel bin). This discontinuity causes gradients to vanish or become undefined (Dirac delta functions) during backpropagation.
• Biased Estimation: Existing solutions either rely on frame-based features (ignoring raw event gradients) or use "surrogate gradients" (e.g., Straight-Through Estimators, Sigmoid relaxations) to approximate the gradient. These surrogates are heuristic and introduce bias, leading to suboptimal learning efficiency and convergence.
• The Gap: There is a lack of a method that allows for unbiased gradient estimation for discontinuous binning operations without altering the forward pass output or relying on heuristic assumptions.

2. Methodology: Functional Backpropagation (FBP)

The authors propose Functional Backpropagation (FBP), a framework that synthesizes unbiased gradients by lifting the binning operation from a discrete space to a functional space.

Core Concept: Integration by Parts

The method leverages the mathematical concept of weak derivatives. While the pointwise derivative of a discontinuous function is undefined (or a Dirac delta), its integral is well-defined.

1. Lifting to Functionals: Instead of treating binning as a discrete map $h: P \to \mathbb{R}^W$, the method treats it as a functional $h[P](x)$ defined over a continuous domain.
2. Cotangent Reconstruction: During the backward pass, the standard backpropagation yields a discrete cotangent vector (gradients at bin centers). FBP interprets this vector as samples of a continuous cotangent function $v_h(x)$.
3. Integration by Parts: By applying integration by parts to the chain rule in functional space, the method avoids evaluating the undefined Dirac delta directly. Instead, it computes the gradient as a convolution:
   $$\tilde{v}_{x'} = \int v_h(x) \frac{\partial h(x)}{\partial x'} dx$$
   Using the property of weak derivatives, this is transformed into:
   $$\tilde{v}_{x'} = \sum_j v_{h_j} \cdot w'_i \cdot \frac{\partial}{\partial x'_i} \kappa\left(\frac{x'_i - j\Delta}{\Delta}\right)$$
   Where $\kappa = l * k$ is the convolution of the binning kernel $k$ and a reconstruction kernel $l$.
4. Synthesis: The method synthesizes a weak derivative kernel $\kappa'$ that approximates the long-range finite differences of the target function. This ensures the gradient is unbiased (provably matching the true finite difference for smooth and non-smooth targets) without changing the forward output.

Algorithm Implementation

• Forward Pass: Remains identical to standard binning (using the original kernel $k$).
• Backward Pass: Replaces the standard gradient accumulation with a convolution using the synthesized kernel $\kappa'$. This can be implemented efficiently in frameworks like PyTorch (via custom VJP) or JAX (via custom JVP).

3. Key Contributions

1. Formalization of Bias: The paper identifies and mathematically formalizes the bias in event-based gradient estimation caused by binning discontinuities.
2. Functional Backpropagation Framework: Introduces a novel method to lift binning to functional space, enabling the computation of unbiased gradients via weak derivatives and integration by parts.
3. Theoretical Guarantees: Derives an exact formula for the synthesized weak derivative, proving its equivalence to long-range finite differences for arbitrary target functions. It demonstrates that the method is unbiased and naturally generalizes to both discontinuous and smooth kernels.
4. Empirical Validation: Extensive experiments show consistent improvements across optimization-based tasks and deep learning pipelines.

4. Experimental Results

The authors evaluated FBP on three distinct tasks:

A. Optimization-Based Motion Estimation

• Task: Estimating angular and linear velocity by optimizing the sharpness of Warped Event Images (IWE).
• Results:
  • Accuracy: Achieved 3.2% lower RMS error compared to standard methods.
  • Convergence: 1.57× faster convergence speed.
  • Bias Analysis: Demonstrated that FBP significantly reduces gradient bias compared to Rect, Linear, and Gaussian binning kernels, even outperforming the "unbiased" Linear kernel by better approximating long-range finite differences.

B. Self-Supervised Optical Flow (MotionPriorCMax)

• Task: Training a state-of-the-art optical flow network on the DSEC dataset using a contrast loss with a discontinuous Rect kernel.
• Results:
  • Achieved 9.4% lower End-Point Error (EPE).
  • Reduced Angular Error and outlier rates.
  • Visualizations showed more robust features and fewer artifacts compared to the baseline.

C. SLAM (CMax-SLAM)

• Task: Event-based Simultaneous Localization and Mapping using Contrast Maximization.
• Results:
  • Achieved 5.1% lower RMS trajectory error.
  • The method preserved correctness on long sequences where the baseline failed due to accumulated errors, demonstrating that sharper IWEs (enabled by unbiased gradients) reduce mapping ambiguity.

5. Significance and Impact

• Bridging the Gap: FBP resolves the long-standing trade-off between using raw event data (which is discontinuous) and differentiable learning. It allows event-based algorithms to fully leverage the temporal resolution of events without relying on heuristic gradient approximations.
• General Applicability: The framework is not limited to event cameras; it offers a theoretical pathway for training neural networks with any discontinuous non-linearity (e.g., Spiking Neural Networks) by providing a mathematically grounded, unbiased gradient estimator.
• Efficiency: Despite the theoretical complexity of functional derivatives, the implementation is efficient (only modifying the backward pass) and yields significant gains in both accuracy and convergence speed, making it a practical drop-in replacement for standard binning in event-based vision pipelines.

In conclusion, this paper provides a rigorous mathematical solution to the gradient bias problem in event-based vision, demonstrating that Functional Backpropagation enables superior performance in both classical optimization and modern deep learning tasks.