Technical Summary: Learning Disentangled Representations for Generalized Multi-view Clustering

1. Problem Statement

Multi-View Clustering (MVC) aims to leverage complementary information from diverse data views (e.g., text, images, omics) to uncover intrinsic structures. However, existing deep MVC methods often suffer from view-distribution entanglement. When fusing representations from different views, variations in data distributions across views lead to misaligned latent spaces. This entanglement results in unclear clustering structures where group clusters are not tight, and the model struggles to distinguish between view-specific noise and genuine shared patterns. Furthermore, existing approaches often fail to learn representations that are simultaneously aligned across views and robust to view-specific noise, leading to suboptimal clustering performance, particularly in incomplete or noisy scenarios.

2. Methodology: Generalized Multi-view Auto-Encoder (GMAE)

The authors propose GMAE, a self-supervised framework designed to learn disentangled representations by decoupling source features into view-specific and view-common embeddings. The framework operates through the following mechanisms:

2.1 Disentangled Representation Learning

GMAE employs dual-path autoencoders to decompose multi-view data:

• Encoders: A view-specific encoder ($f^v_{\theta_v}$) and a cross-view shared encoder ($f_{\theta_c}$) process the input.
• Decomposition: Features are mapped into view-specific representations ($Z^v$) and view-common representations ($C^v$).
• Reconstruction: A decoder reconstructs the input using the concatenation of specific and common features ($[Z^v \oplus C^v]$).
• Objective: The framework maximizes the mutual information (MI) between the original view and its reconstruction (minimizing reconstruction loss $L^v_{Rec}$) while minimizing the MI between view-specific and view-common representations to enforce statistical independence. This is achieved via a correlation loss ($L^v_{Cor}$), which minimizes the covariance between $Z^v$ and $C^v$.

2.2 Cross-View Adversarial Alignment

To address distribution misalignment, GMAE introduces a cross-view adversarial mechanism:

• Discriminators: View-specific discriminators ($D^v_{\kappa_v}$) are trained to distinguish whether a representation originates from the target view (real) or a different view (fake).
• Alignment: Through a min-max adversarial game, the view-specific encoders are guided to produce representations that are indistinguishable across views regarding their distribution, thereby aligning the view-specific distributions while preserving their unique characteristics. This is formalized as a generative adversarial loss ($L^v_{Dis}$).

2.3 Self-Correlation and Clustering Structure

To enhance the clustering capability without labels, GMAE maximizes the mutual information between the comprehensive representation ($Q$) and its local neighborhood ($Q_\omega$).

• Contrastive Objective: Using an InfoNCE-inspired approach, the model maximizes the MI between a sample and its nearest neighbors to promote intra-cluster cohesion and inter-cluster separation. This is optimized via a cross-entropy loss ($L_{Ent}$).
• Theoretical Guarantee: The authors provide theoretical proofs (Theorems 1-3) demonstrating that under the proposed constraints, the learned common representations contain all necessary clean information and that the comprehensive representation preserves more cluster-relevant information and less cluster-irrelevant noise compared to standard contrastive learning methods.

2.4 Objective Function

The total objective function combines these components:
$$J = \sum_{v=1}^{V} L^v_{Rec} + \alpha \times (L^v_{Cor} + L^v_{Dis}) + \beta \times L_{Ent}$$
where $\alpha$ and $\beta$ are hyperparameters balancing the regularization terms.

3. Key Contributions

The paper outlines three primary contributions:

1. Novel Framework: GMAE is presented as the first self-supervised disentangled representation learning framework specifically addressing the challenge of view distribution and inter-cluster entanglement in MVC.
2. Theoretical Analysis: The authors rigorously derive an optimizable loss function based on mutual information estimation. They provide a theoretical analysis showing that GMAE's learned representations contain significantly more cluster-relevant information and less noise compared to existing methods.
3. Empirical Superiority: The framework simultaneously improves clustering consistency across views and preserves view diversity. It demonstrates robustness in both complete and incomplete multi-view clustering tasks across diverse data types (text, image, omics, synthetic).

4. Experimental Results

The authors evaluated GMAE on 13 benchmark datasets, including synthetic data, image datasets (STL-10, Digits, LandUse-21, etc.), image-text pairs (Wikipedia, RGB-D), and multi-omics data (BRCA, LGG).

• Performance: GMAE consistently outperformed 12 state-of-the-art (SOTA) methods (e.g., COMPLETER, DCP, DealMVC, SCMVC) in terms of Accuracy (ACC), Normalized Mutual Information (NMI), and Purity (PUR). It achieved 15 first-place, 3 second-place, and 2 third-place finishes across 20 tests.
• Incomplete Data: In experiments with missing views (missing rates up to 0.9), GMAE exhibited superior resilience, showing a significantly slower rate of performance decline compared to competing methods.
• Scalability: On the Digits dataset, GMAE's performance improved as the number of views increased, whereas other methods (like DSMVC and SURE) suffered from performance degradation with additional views.
• Stability: The model demonstrated stable training convergence without the significant fluctuations observed in other methods (e.g., CPSPAN, DCP).
• Ablation Studies: Experiments confirmed that the combination of reconstruction, correlation/adversarial, and self-correlation losses is necessary for optimal performance; removing any component led to significant metric drops.

5. Significance and Claims

The paper claims that GMAE addresses a fundamental limitation in current deep MVC: the inability to align view distributions without losing view-specific discriminative power or introducing entanglement. By explicitly disentangling view-specific and view-common factors and aligning them via adversarial learning, GMAE generates robust, non-trivial embeddings.

The authors assert that this approach provides a scalable solution with linear computational complexity relative to dataset size, making it suitable for large-scale data. The framework is particularly significant for real-world applications where data is often incomplete, noisy, or heterogeneous, as it maintains high clustering quality even under adverse conditions where traditional methods fail. The code is made available to facilitate reproducibility and further research.