Model--based clustering for spherical and hyper--spherical data using elliptically symmetric distributions

Technical Summary: Model–based clustering for spherical and hyper–spherical data using elliptically symmetric distributions

Problem Statement
Directional data, constrained to a unit norm ($S^d$), is prevalent in fields such as geology, environmental sciences, and robotics. While model-based clustering for such data has attracted significant interest, most existing methods rely on rotationally symmetric distributions (e.g., von Mises–Fisher, Projected Normal, Spherical Cauchy). These distributions assume isotropic covariance, which restricts their ability to model complex data structures where clusters exhibit elliptical shapes rather than circular or spherical symmetry. Although elliptically symmetric distributions like the Kent distribution have been proposed, there is a need for robust mixture models utilizing recently proposed distributions that offer computational advantages and flexible contour shapes.

Methodology
The paper proposes model-based clustering for spherical and hyper-spherical data using mixtures of two specific elliptically symmetric distributions:

1. Elliptically Symmetric Angular Gaussian (ESAG): Proposed by Paine et al. (2018), this distribution arises from projecting a multivariate normal distribution onto the sphere. It imposes constraints on the covariance matrix to ensure identifiability, allowing for elliptical contours that match the Fisher–Bingham distribution but with a closed-form normalizing constant.
2. Spherical Elliptically Symmetric Projected Cauchy (SESPC): Proposed by Tsagris and Alzeley (2025), this distribution is derived by projecting a 3-dimensional Cauchy distribution onto the sphere under similar elliptical symmetry constraints.

Estimation and Implementation:

• Algorithm: The Expectation–Maximization (EM) algorithm is employed to estimate the parameters of the finite mixture models. The E-step computes posterior membership probabilities, while the M-step updates mixing proportions and distribution parameters.
• Optimization: Since closed-form maximizers for the mean ($\mu$) and shape ($\gamma$) parameters do not exist for these densities, numerical optimization (e.g., Nelder–Mead) is used in the M-step.
• Model Selection: The Integrated Completed Likelihood (ICL) criterion is utilized to determine the optimal number of clusters ($K$).
• Initialization: To avoid local maxima, the authors compare initialization via $k$-means and Gaussian Mixture Models (GMMs), finding that GMMs with structured covariance yield better performance for these elliptical distributions.
• Covariates: The framework allows for the inclusion of covariates to model mixing probabilities (concomitant model) or mean vectors (mixture of regressions), though this increases computational cost.
• Hyper-spherical Data: For data residing on hyper-spheres ($d > 2$), the authors project the data onto a 3-dimensional sphere using Principal Geodesic Analysis (PGA) before applying the spherical mixture models.

Key Contributions

• Extension of Clustering Frameworks: The paper extends model-based clustering beyond rotationally symmetric distributions to include ESAG and SESPC, enabling the modeling of clusters with elliptical contours.
• Comparative Analysis: The study provides a comprehensive comparison between ESAG and SESPG mixtures regarding cluster detection accuracy, robustness to cluster shape, and computational efficiency.
• Hyper-spherical Adaptation: It addresses the application of these spherical models to hyper-spherical data via projection, a necessary step given that SESPC was originally defined for 3D spheres.

Results

• Simulation Studies:
  • Accuracy: When data is generated from ESAG mixtures, both models perform similarly well. However, when data is generated from SESPC mixtures, the SESPC model outperforms ESAG, particularly in detecting the correct number of clusters and achieving higher Adjusted Rand Index (ARI) values.
  • Cluster Shape: The SESPC distribution is noted to be more "spread" in the tails compared to ESAG. Consequently, ESAG struggles more when the true data structure is highly dispersed or when clusters are formed at specific angles.
  • Sample Size and Concentration: Performance improves with larger sample sizes and higher concentration parameters ($\tau$). Small sample sizes or low concentration parameters lead to difficulties in detecting the correct number of clusters.
  • Computational Cost: The SESPC model is consistently faster than the ESAG model across various sample sizes ($n=200, 500, 1000$).
• Real Data Applications:
  • Earthquake Data (North America & Fiji): Both models identified similar numbers of clusters for North American data (4 clusters), though they assigned specific locations (e.g., Hawaii) differently. SESPC clusters were found to be more linearly separable. For the Fiji region, SESPC selected 4 clusters while ESAG selected 7; SESPC clusters were again easier to separate. The inclusion of earthquake magnitude as a covariate had a weak effect on North American data but reduced the number of clusters for the Fiji ESAG model.
  • Wine Quality Data: Both models correctly identified 2 clusters. ESAG achieved a higher ARI (0.700) compared to SESPC (0.546), suggesting ESAG was more appropriate for this specific dataset.
  • Wholesale Data: ESAG detected 3 clusters while SESPC detected 2.
  • Comparison with SC: In both real-world hyper-spherical datasets (Wine and Wholesale), the proposed elliptical models outperformed the rotationally symmetric Spherical Cauchy (SC) mixture model in terms of ARI.

Significance and Claims
The authors claim that the primary significance of this work lies in relaxing the isotropic covariance assumption inherent in most directional clustering methods. By utilizing ESAG and SESPC mixtures, the proposed models can capture ellipsoidal cluster structures, offering a more flexible alternative to rotationally symmetric distributions.

The paper modestly concludes that while both distributions are effective, the choice between them depends on the underlying data generation process: ESAG is suitable for data with tighter, less dispersed clusters, while SESPC is superior for data with heavier tails or more dispersed structures. Furthermore, the study demonstrates that projecting hyper-spherical data onto a sphere allows these spherical mixture models to be effectively applied to higher-dimensional directional data, outperforming traditional rotationally symmetric approaches in empirical tests.