Technical Summary: Relational Retrieval for Generalized Category Discovery

Problem Definition

The paper addresses Generalized Category Discovery (GCD), a task that extends traditional classification to open-world scenarios. The objective is to jointly recognize known classes using labeled data ($D_L$) and discover novel categories within unlabeled data ($D_U$). A critical challenge in existing GCD approaches is that they typically treat labeled and unlabeled data as separate learning streams: labeled data undergoes supervised learning, while unlabeled data relies on self-supervised signals. This separation leads to two primary limitations identified by the authors:

1. Wasted Semantic Knowledge: Unlabeled samples belonging to known classes rely solely on self-supervision, ignoring the highly discriminative features already captured by labeled samples of the same classes.
2. Ineffective Discovery: Category discovery often proceeds without considering relational patterns. Samples from the same novel category, while distinct from known classes, maintain consistent relationships with known-class prototypes, a signal often overlooked in favor of unreliable pseudo-labeling.

Methodology: Relational Pattern Consistency (RPC)

The proposed framework, Relational Pattern Consistency (RPC), explicitly couples labeled and unlabeled data through bidirectional knowledge transfer. The method builds upon the SimGCD baseline for representation learning and incorporates DebGCD's One-vs-All (OVA) decomposition to identify whether unlabeled samples belong to known (ID) or novel (OOD) classes.

The core of RPC consists of two mechanisms:

1. Known-Class Preservation via Semantic Transfer

To prevent the loss of semantic structure in unlabeled known-class samples, RPC transfers robust patterns from labeled to unlabeled data.

• Soft ID/OOD Decomposition: OVA classifiers generate soft weights ($w_{old}$ and $w_{new}$) to estimate the likelihood of a sample belonging to known or novel classes.
• Confidence-Weighted Embedding Fusion: For batches containing both labeled and high-confidence unlabeled (ID) samples, the model constructs a fusion matrix. This matrix performs a circular shift to inject information from labeled samples into their paired unlabeled counterparts. The fusion is weighted by $w_{old}$, ensuring only high-confidence ID samples are significantly influenced.
• Behavioral Alignment: The framework enforces consistency between the augmentation-induced behavioral changes ($\Delta$) of labeled samples and their fused unlabeled counterparts. This ensures unlabeled known-class samples inherit robust representations from labeled data.

2. Category Discovery via Relational Matching

For novel category discovery, RPC moves beyond ill-posed clustering by leveraging relational consistency.

• Relational Signatures: The method posits that samples from the same novel category exhibit invariant relationships with known-class prototypes. Specifically, two samples from the same novel class should have similar distances (similarity patterns) to all known-class prototypes.
• Pattern Matching: Each sample is encoded as a vector of similarities to known-class prototypes ($r(x)$). The model then minimizes the distance between these relational signatures for feature-similar samples, weighted by their novelty scores ($w_{new}$). This transforms the discovery task into a well-defined pattern matching problem within the known-class reference frame.

The total training objective combines the baseline losses (representation and classifier) with the new alignment loss ($L_{align}$) and the novel discovery loss ($L_{new}$).

Key Contributions

The authors outline three primary contributions:

1. Bidirectional Coupling: RPC is the first GCD framework to explicitly couple labeled and unlabeled data through bidirectional knowledge transfer, utilizing soft ID/OOD decomposition to enable targeted interactions rather than maintaining disjoint learning streams.
2. Novel Mechanisms:
   • Introduction of confidence-weighted embedding fusion with behavioral alignment to preserve known-class semantics.
   • Development of relational consistency learning, which discovers novel categories based on their invariant relationships with known-class prototypes, avoiding reliance on unreliable pseudo-labels.
3. State-of-the-Art Performance: Extensive experiments demonstrate that RPC achieves superior performance on both generic and fine-grained benchmarks, with consistent improvements in both known (Old) and novel (New) class accuracy.

Experimental Results

The method was evaluated on six benchmarks: three fine-grained datasets (CUB, Stanford Cars, FGVC-Aircraft) and three general datasets (CIFAR-10, CIFAR-100, ImageNet-100).

• Performance: RPC achieved state-of-the-art results across most benchmarks. Notably, on the CUB dataset, it improved the "All" accuracy to 67.1% (vs. 66.3% for the previous best, DebGCD), with significant gains in "Old" accuracy (73.2% vs. 71.8%) and "New" accuracy (64.8% vs. 63.5%).
• Ablation Studies: Removing the embedding fusion or the alignment loss ($L_{align}$) resulted in a drop of approximately 3% in known-class accuracy on CUB. Removing the relational discovery loss ($L_{new}$) caused a 3.3% drop in novel-class accuracy, confirming the necessity of both components.
• Efficiency: The method introduces minimal computational overhead, adding only 1.40% to the parameter count and 1.82% to FLOPs compared to the SimGCD baseline.
• Visualization: t-SNE visualizations indicate that RPC produces a more structured latent space with tighter intra-class grouping and clearer inter-class boundaries compared to baseline methods, effectively reducing semantic entanglement between neighboring categories.

Significance and Claims

The paper claims that RPC fundamentally shifts the GCD paradigm from parallel learning (where labeled and unlabeled data are processed separately) to mutual enhancement. By leveraging the structured interactions between known and novel data, the framework allows labeled data to guide the representation learning of unlabeled known classes while simultaneously using the collective relational signatures of unlabeled data to discover novel categories. The authors assert that this approach validates the hypothesis that labeled and unlabeled data can mutually enhance each other through structured interactions, leading to more robust feature representations and more accurate category discovery in open-world settings.