Technical Summary: ANGLE - Angular Neural Generative Learning via Engression

Problem Statement

Circular data, representing angles or directions on the unit circle $S^1$ (e.g., wind directions, object poses, times of day), are prevalent in fields such as computer vision, meteorology, and biology. Traditional regression approaches typically target the conditional mean of an angular response. However, for circular data, the conditional distribution is frequently multimodal, skewed, or asymmetric. In such cases, a single point summary (like the mean) is geometrically misleading and fails to capture the full structure of the relationship between covariates and the response. Furthermore, existing methods often lack the ability to provide robust uncertainty quantification, extrapolate to unseen covariate regions, or perform advanced statistical tasks like sufficient dimension reduction (SDR) and conditional distribution equality testing within a unified framework.

Methodology

The paper introduces ANGLE (Angular Neural Generative Learning via Engression), a lightweight deep generative framework designed for non-parametric distributional regression on circular data.

Core Framework

Instead of explicitly modeling a parametric density, ANGLE learns the conditional distribution $P(Y|X)$ implicitly through a generative map $g(X, \epsilon)$, where $X \in \mathbb{R}^d$ are covariates and $\epsilon \sim U(0,1)$ is latent noise. The model assumes $Y | X \stackrel{d}{=} g(X, \epsilon)$. The framework accommodates both pre-additive noise models (noise injected before the non-linear link) and post-additive noise models (noise injected after).

Loss Function: Generalized Circular Energy Score (GCES)

To optimize the generative map, ANGLE minimizes a Generalized Circular Energy Score (GCES) loss. Standard Euclidean energy scores fail on circular data due to the periodic wrapping effect ($0 \equiv 2\pi$). The proposed loss is defined as:
$$\text{ES}(P, \theta) = \mathbb{E}[F(d(\theta, \Theta))] - \frac{1}{2}\mathbb{E}[F(d(\Theta, \Theta^*))]$$
where $\Theta, \Theta^*$ are independent copies from distribution $P$, $d$ is a circular distance metric (geodesic or chordal), and $F$ is a continuous function chosen such that the loss is a strictly proper scoring rule. The paper establishes conditions under which GCES is strictly proper, ensuring that the expected penalty is minimized if and only if the predicted distribution matches the true distribution.

Architecture and Flexibility

The model utilizes a Multi-Layer Perceptron (MLP) backbone with batch normalization and residual connections. It supports:

• Input Handling: Native support for both Euclidean and circular covariates (via trigonometric embeddings).
• Output Variants: Three transformation strategies to map unconstrained network outputs to $[0, 2\pi)$:
  1. Biternion: Output $(u, v) \in \mathbb{R}^2$ mapped via $\text{atan2}(v, u)$.
  2. Scaled Sigmoid: Output bounded strictly within $[0, 2\pi)$.
  3. Unbounded Modulo: Unbounded scalar output mapped via modulo $2\pi$ (wrapped modeling).
• Noise Injection: Stochastic noise can be injected at the input (pre-ANM) or hidden layers (post-ANM).

Key Contributions

The paper outlines four primary contributions:

1. ANGLE Framework and Theoretical Foundations:

   • Proposes a deep generative distributional regression method for circular data.
   • Establishes population-level guarantees (Theorem 1) showing that the minimizer of the GCES loss recovers the true conditional distribution.
   • Proves rotational equivariance (Lemma 1) for derived estimators (mean, median, mode, density), ensuring invariance to the choice of reference direction.
   • Characterizes conditions for the strict propriety of the GCES loss.

2. Unified Toolkit for Downstream Tasks:
   The framework enables a suite of statistical tasks with theoretical guarantees:

   • Point Prediction with Uncertainty Quantification: Estimation of circular mean, median, mode, density, dispersion, and quantiles via forward simulation.
   • Sufficient Dimension Reduction (SDR): Recovery of the central sigma field $\mathcal{G}^*_{Y|X}$ by estimating the index parameter $\beta$ (Theorems 2–4).
   • Extrapolation: Under assumptions of real analyticity and identifiability, the framework provides guarantees for distributional extrapolation beyond the training support (Theorem 5).
   • Conditional Distribution Equality Testing: A procedure (Algorithm 1) to test $H_0: P^{(1)}_{Y|X} = P^{(2)}_{Y|X}$ using synthetic data generation and circular two-sample tests.

3. Architectural Flexibility and Efficiency:
   The framework is computationally lightweight compared to Bayesian alternatives and classical non-parametric methods, while accommodating both pre- and post-additive noise structures.

4. Empirical Validation:
   Extensive simulations and real-world applications demonstrate efficacy across image data (object pose estimation) and tabular data (wind direction prediction).

Results

Simulations

• Predictive Performance: ANGLE consistently outperforms baselines (including Circular Linear Regression, Kernel CLR, SCBR, and MCR2) across 16 simulation settings involving linear and non-linear relationships, as well as Euclidean and circular covariates.
• Metrics: It achieves the lowest Circular Continuous Ranked Probability Score (CRPS) and Mean Absolute Angular Deviation (MAAD).
• Extrapolation: In extrapolation scenarios, ANGLE successfully preserves structural trends outside the training support, whereas baseline methods (like Kernel CLR) fail to extrapolate and asymptote.

Real-World Applications

1. Object Pose Estimation (PASCAL3D+):

   • Applied to 12 object categories using Inception-v3 and ConvNeXt visual encoders.
   • ConvNeXt-ANGLE achieved the best mean scores across all metrics (Accuracy$_{\pi/6}$, CRPS, MedErr, MAAD) and was statistically superior to parametric mixture models and other baselines.
   • The method provided intrinsic uncertainty quantification, generating 95% prediction intervals that correctly widened for ambiguous inputs (e.g., occluded or symmetric objects).

2. Wind Direction Prediction:

   • Evaluated on German and Indian meteorological datasets.
   • ANGLE achieved superior CRPS scores on both datasets.
   • Computational Efficiency: On the German dataset, ANGLE completed training and inference in ~2 seconds, compared to ~21,677 seconds for the Bayesian vMQP baseline, representing a 99.99% reduction in computational overhead.
   • Extrapolation Test: On the Indian dataset with a spatial block-split (training on the southwest, testing on the northeast), ANGLE significantly outperformed baselines in out-of-distribution (OOD) settings, despite the lack of strict monotonicity in the underlying meteorological patterns.

3. Sufficient Dimension Reduction (SDR):

   • Applied to the PASCAL3D+ dataset, reducing 2048-dimensional features to a 5-dimensional subspace.
   • The reduced model maintained high predictive performance, outperforming all baselines and demonstrating the efficacy of the SDR capability.

4. Conditional Distribution Equality Testing:

   • Used to test if azimuth distributions differ across object classes.
   • Results confirmed that structurally similar objects (e.g., car/bus) share similar conditional distributions, while distinct objects showed significant divergence, validating the method's utility in exploratory analysis.

Significance and Claims

The paper claims that ANGLE provides a unified, lightweight, and theoretically grounded framework for distributional learning on circular data. Its significance lies in:

• Moving beyond point estimates: It addresses the geometric limitations of mean-based regression by modeling the full conditional distribution, which is crucial for multimodal and asymmetric circular responses.
• Bridging the gap in circular statistics: It extends the "Engression" concept (originally for Euclidean data) to the circular domain, solving challenges related to periodicity and wrapping.
• Practical Utility: It offers a single toolbox for prediction, uncertainty quantification, dimension reduction, extrapolation, and hypothesis testing, all while remaining computationally efficient enough for large-scale applications like video analysis or real-time wind forecasting.
• Theoretical Rigor: The work establishes population-level guarantees, rotational equivariance, and conditions for extrapolability, filling a theoretical void in non-parametric circular regression.

The authors note that while the framework is robust, future work could extend it to more general non-Euclidean spaces (spheres, tori) and probabilistic forecasting of circular time series.