Technical Summary: Subspace Clustering on Incomplete Data with Self-Supervised Contrastive Learning

Problem Definition

Subspace clustering aims to partition high-dimensional data points lying in a union of low-dimensional subspaces. While effective in domains such as gene expression analysis, hyperspectral imaging, and recommendation systems, existing methods predominantly assume fully observed data. In real-world scenarios, data is frequently incomplete due to occlusion, sensor failure, or resource constraints. Traditional approaches, which rely on pairwise affinities or matrix decompositions, struggle with missing entries, often yielding suboptimal results when naive imputation is used or failing to scale with large datasets. Furthermore, many deep learning-based subspace clustering methods also assume complete inputs, overfitting to observed entries or requiring biased imputation strategies.

Methodology: Contrastive Subspace Clustering (CSC)

The authors propose Contrastive Subspace Clustering (CSC), a self-supervised framework designed to learn robust embeddings from incomplete data without requiring labels or explicit data reconstruction.

Core Framework

The pipeline operates in two main stages: training and inference.

1. Data Augmentation via Random Masking:
   For each partially observed sample $y_i = M_i \odot x_i$ (where $M_i$ is a binary mask), the method generates two disjoint augmented views, $y_i^a$ and $y_i^b$, by applying two disjoint random masks ($M_i^a$ and $M_i^b$) to the original incomplete sample. These views serve as positive pairs for contrastive learning.

2. Contrastive Learning (SimCLR-style):
   A shared deep neural network (backbone) $f_\theta$ processes both augmented views to produce latent representations $h_i^a$ and $h_i^b$. These are projected via a head $g_\phi$ into a latent space. The model is trained using a Normalized Temperature-scaled Cross-Entropy (NT-Xent) loss. This objective pulls the embeddings of the two views from the same sample together while pushing apart embeddings from different samples. This encourages the network to learn invariant features that capture the underlying subspace structure despite missing entries.

3. Inference and Clustering:
   After training, the projection head is discarded. The trained backbone $f_\theta$ embeds the original incomplete samples into a latent space. These embeddings are then fed into a standard Sparse Subspace Clustering (SSC) algorithm to recover the final cluster assignments. Notably, the method does not reconstruct the missing data; it directly clusters the learned representations.

Key Contributions

• Novel Framework for Incomplete Data: CSC is the first method to apply a contrastive self-supervised framework specifically for subspace clustering with missing data. It treats random masking as a data augmentation strategy rather than a problem to be solved via imputation.
• Robustness to Missingness: By learning invariance across masked views, the method captures subspace structures even when individual feature values are absent, avoiding the artifacts introduced by two-stage imputation methods.
• Scalability: Unlike optimization-based methods that scale quadratically with the number of samples (due to affinity matrix construction), CSC utilizes a forward pass for inference, offering near-instantaneous clustering and significantly reduced computational demands.
• Modularity: The approach decouples representation learning from the clustering algorithm, allowing the learned embeddings to be used with any standard clustering technique (e.g., k-means, spectral clustering, or SSC).

Experimental Results

The authors evaluated CSC on six benchmark datasets, including standard computer vision datasets (MNIST, COIL20) and hyperspectral imaging (HSI) datasets (IndianPines, Pavia University, Salinas).

• Performance vs. Baselines: CSC consistently outperformed both classical methods (e.g., EWZF-SSC, GSSC, MSC) and deep learning baselines (e.g., DSC, MAE, DMFMC).
  • On MNIST, CSC outperformed all benchmarks by more than 15% in mean accuracy.
  • On Pavia and IndianPines (HSI datasets), improvements of over 5% and 6% were observed, respectively.
  • On datasets with high missing rates (low sampling rates $\rho$), CSC maintained stable performance, whereas classical methods and Masked Autoencoders (MAE) suffered significant degradation, particularly under high noise conditions.
• Ablation Studies:
  • Model Depth: Increasing network depth improved accuracy up to a point (plateauing around depth 5), indicating the benefit of model expressiveness.
  • Residual Connections: The choice of residual connection strategy (Full, Block, or None) had a minor impact on performance, with "No Residual" performing slightly better on large datasets.
  • Noise Robustness: CSC demonstrated superior robustness compared to MAE under both low and high noise regimes, maintaining low clustering errors even when sampling rates were moderate-to-low ($\rho \approx 0.5$).

Significance and Claims

The paper claims that CSC offers a scalable, robust, and label-free solution for subspace clustering in the presence of missing data. Its significance lies in:

1. Overcoming the Imputation Bottleneck: By avoiding explicit reconstruction, the method sidesteps the distortion of subspace geometry often caused by imputation steps in high missing-rate scenarios.
2. Efficiency: The ability to perform inference via a simple forward pass makes the method significantly more efficient than traditional optimization-based subspace clustering methods.
3. Generalizability: The framework is applicable to diverse data modalities, including standard images and high-dimensional hyperspectral data, without requiring domain-specific architectural changes beyond the backbone selection.

The authors conclude that while subspace clustering with missing data remains a challenging problem, particularly in high-dimensional regimes, the proposed contrastive approach provides a strong foundation for future work in streaming and multi-view settings.