Technical Summary: GICDM – Mitigating Hubness for Reliable Distance-Based Generative Model Evaluation

1. Problem Statement

The evaluation of generative models frequently relies on high-dimensional embedding spaces (e.g., InceptionV3, DINOv2, CLAP) to compute distances between real and synthetic samples. However, these high-dimensional spaces are susceptible to the hubness phenomenon, a consequence of the curse of dimensionality.

In high dimensions, nearest-neighbor relationships become highly asymmetric. Certain points, termed hubs, appear disproportionately often as the nearest neighbors of other points, even when they are semantically unrelated. Conversely, many points, termed antihubs, never appear as nearest neighbors. This distortion invalidates the assumption that proximity implies semantic similarity.

The paper demonstrates that hubness severely biases standard distance-based fidelity and coverage metrics (e.g., Precision, Recall, Density, Coverage). Specifically:

• False Fidelity: Generated "hub" points artificially appear close to many real points, inflating fidelity scores.
• False Coverage: Real points in "antihub" regions act as blind spots, receiving zero fidelity scores from generated samples.
• Metric Failure: In scenarios where real and generated distributions are disjoint (e.g., mixtures of hyperspheres with swapped radii), standard metrics incorrectly yield non-zero scores in high dimensions, failing to detect the lack of overlap.

Existing metrics often assume normality or rely on simple distributional approximations that do not account for these structural distortions in the embedding space.

2. Methodology: GICDM

The authors propose Generative ICDM (GICDM), a method designed to mitigate hubness specifically for the evaluation of generative models. GICDM extends the classical Iterative Contextual Dissimilarity Measure (ICDM) (Jégou et al., 2010), which was previously dismissed for generative evaluation due to computational concerns and lack of out-of-sample handling.

GICDM operates under three key desiderata:

1. Uniformize Real Data Density: Reduce hubness in the real dataset so that nearest-neighbor relationships are meaningful and symmetric.
2. Preserve Relative Positioning: Maintain the true local geometric structure between generated points and the real data manifold.
3. Conditional Independence: Ensure the evaluation of a generated point depends solely on its relationship to the real set, not on other generated points.

Algorithmic Steps

1. ICDM on Real Data: The ICDM algorithm is applied iteratively to the set of real data points $\{x^r_i\}$. This process computes scaling factors $\{\delta^r_i\}$ that effectively flatten the local density of the real data manifold. The update rule iteratively scales distances to equalize the average neighbor distance for all points, converging to a state where local density estimates are uniform.
2. Out-of-Sample Scaling for Generated Points: For each generated point $x^g_j$, a scaling factor $\delta^g_j$ is computed independently. This is derived by enforcing that the average neighbor distance of the generated point (relative to the scaled real neighbors) matches the equilibrium average distance of the real data.
   $$\delta^g_j = \frac{\mu_{r,eq}}{\frac{1}{K+1} \sum_{k=1}^{K+1} \delta^r_{NN^{eq}_k(x^g_j)} d(x^g_j, NN^{eq}_k(x^g_j))}$$
   Here, $K+1$ is used instead of $K$ to account for the fact that the generated point is not part of the real set.
3. Multi-Scale Filtering: To prevent overcorrection, the method employs a filtering strategy. If a generated point's computed scaling factor $\delta^g_j$ deviates significantly from the average scaling of its real neighbors, it is flagged as being outside the real data manifold.
   • The filtering is performed at multiple scales (using two different neighborhood sizes, e.g., $K_1$ and $K_2$). This addresses the "crossover regime" where, in specific dimensions and densities, out-of-manifold points might coincidentally have distances similar to in-manifold points.
   • Points failing the filter are assigned zero fidelity and do not contribute to coverage.

The final dissimilarity between a real point $x^r_i$ and a generated point $x^g_j$ is defined as:
$$\text{GICDM}(x^r_i, x^g_j) = d(x^r_i, x^g_j) \cdot \delta^r_i \cdot \delta^g_j$$

3. Key Contributions

• Demonstration of Hubness in Embeddings: The paper empirically shows that common embedding spaces used for generative model evaluation (VGG16, InceptionV3, DINOv2, DINOv3, CLAP) exhibit significant hubness, with top 1% of points appearing 4–16 times more frequently as neighbors than average, and 10–30% of points acting as antihubs.
• Revival of ICDM: The authors demonstrate that the Iterative Contextual Dissimilarity Measure (ICDM), previously overlooked in favor of the non-iterative NICDM, consistently outperforms other hubness reduction methods (including CSLS, DSL, and Sphere projection) across 17 dataset-embedding combinations.
• GICDM Framework: The introduction of a tailored method that applies ICDM to real data while handling out-of-sample generated points via independent scaling and multi-scale filtering.
• Benchmark Validation: Extensive testing on synthetic benchmarks (Räsä et al., 2025) and real-world datasets (CIFAR-10, ImageNet, LSUN, FFHQ) showing that GICDM resolves metric failures caused by hubness.

4. Experimental Results

• Synthetic Benchmarks: On the benchmark proposed by Räsä et al. (2025), GICDM significantly improved the performance of standard metrics. For example, Clipped Density passed 23 criteria (up from 18) and Clipped Coverage passed 22 (up from 18). Crucially, GICDM corrected the "Hypersphere Test," where disjoint distributions previously yielded non-zero scores due to hubness; with GICDM, scores correctly remained at zero.
• Human Correlation: When evaluated against human error rates (discriminating real vs. generated images), GICDM maintained or improved the Pearson correlation of Clipped Density (the best-performing metric). This improvement was most pronounced for DINOv3 embeddings, which are heavily affected by hubness.
• Classifier-Free Guidance: In experiments with DiT-XL-2 on ImageNet, standard metrics incorrectly suggested that classifier-free guidance increased both fidelity and coverage. GICDM corrected this, showing increased fidelity but decreased coverage, aligning with theoretical expectations.
• Qualitative Analysis: Visualizations of LSUN Bedroom data showed that the most frequent generated hub appeared in 64 real neighborhoods before correction, dropping to 4 after GICDM. The remaining neighbors were semantically more similar to the hub, indicating restored neighborhood structure.

5. Significance and Limitations

The paper claims that GICDM enables more reliable and trustworthy evaluation of generative models in high-dimensional spaces by resolving failures induced by the hubness phenomenon. It restores the validity of distance-based metrics, ensuring they align better with human assessment and theoretical expectations.

Modest Claims and Limitations:

• Dependency on Robustness: The authors explicitly state that GICDM's effectiveness relies on the underlying metric being robust to real outliers. Metrics like Clipped Density and Clipped Coverage, which limit the influence of individual points, benefit most. Non-robust metrics (e.g., standard Precision, Density) may not improve or may even degrade in correlation with human scores when GICDM is applied.
• Not a Distance Metric: GICDM is a dissimilarity measure that does not obey the triangle inequality. However, the authors argue this is acceptable for generative evaluation, which primarily requires ranking neighbors and determining neighborhood membership, operations that do not strictly require the triangle inequality.
• Computational Cost: While GICDM adds overhead (iterative ICDM and scaling calculations), the complexity is $O(N(TN + M))$, which is comparable to the brute-force $O(N^2)$ distance calculations already required for high-dimensional nearest-neighbor searches.

In conclusion, the paper posits that hubness is a fundamental, often overlooked cause of metric failure in generative model evaluation and that GICDM provides a necessary correction to restore the reliability of these evaluations.