Optimal Demixing of Nonparametric Densities

1. Problem Formulation

The paper addresses the problem of linear unmixing of nonparametric densities. The authors consider a setting with $n$ groups of samples, where the $i$-th group consists of $N_i$ independent samples drawn from a density $f_i(x)$. The core assumption is that each $f_i(x)$ is a convex combination of $K$ unknown base (or "archetype") densities $g_1(x), \dots, g_K(x)$:

$$f_i(x) = \sum_{k=1}^K \pi_i(k) g_k(x)$$

where:

• $g_k(x)$ are nonparametric densities on $\mathbb{R}^d$ (typically assumed to belong to a Nikol'ski smoothness class with parameter $\beta$).
• $\pi_i = (\pi_i(1), \dots, \pi_i(K))^\top$ is a mixing weight vector lying in the probability simplex (i.e., $\pi_i(k) \ge 0, \sum \pi_i(k) = 1$).
• The goal is to estimate the base densities $g_1, \dots, g_K$ given the observed samples $\{X_{ij}\}$, without knowing the mixing weights $\Pi = [\pi_1, \dots, \pi_n]^\top$.

Motivation: This problem generalizes discrete topic modeling to continuous variables. Applications include:

• LLMs with Word Embeddings: Estimating the distribution of word embeddings for abstract topics.
• Archetypal Analysis: Expressing distributional data as convex combinations of latent archetypes.
• Decontamination: Recovering base distributions in mutual contamination models.

2. Limitations of Existing Approaches

The authors argue that classical methods fail in this specific setting:

• Standard Kernel Density Estimation (KDE): Applying KDE to each group separately estimates $f_i$ but does not solve for the shared $g_k$. The convergence rate would depend on $N_i$ (per-group sample size) rather than the total sample size $nN$, which is suboptimal.
• Parametric Mixture Models (EM/VAE): These require $g_k$ to have a specific parametric form, which contradicts the nonparametric setting.
• Katz-Samuels et al. (2019): While they address a similar problem, their method relies on strong conditions (degenerate mixing weights) and lacks explicit convergence rates for smooth densities.
• Austern et al. (2025): They propose a two-step method (histogram-based topic modeling followed by kernel smoothing). However, their method suffers from a bias-variance trade-off in the bin size $M$ that prevents it from achieving the minimax optimal rate when the smoothness $\beta > 1$.

3. Methodology: Topic-Weighted Kernel Density Estimator

The proposed solution is a novel estimator that combines topic modeling with a de-biased kernel density estimator. The method proceeds in three conceptual stages:

A. Induced Topic Model

The continuous densities are discretized into a histogram. By partitioning $\mathbb{R}^d$ into $M$ bins $\{B_m\}$, the problem is transformed into a discrete topic model:
$$Y_i \sim \text{Multinomial}(N_i, \Omega_i), \quad \text{where } \Omega_i = \sum_{k=1}^K \pi_i(k) g_k^B$$
Here, $Y_i$ is the count vector of samples in each bin, and $g_k^B$ represents the probability mass of the $k$-th base density in the bins. This allows the use of existing topic modeling algorithms to estimate the "topic matrix" $G$ (the discretized $g_k$) and the mixing weights $\Pi$.

B. Oracle Estimator (Ideal Case)

If the mixing weights $\Pi$ were known, the authors derive an "oracle" estimator for $g_k(x)$ as a weighted sum of the group-specific KDEs:
$$\hat{g}_k^{\text{oracle}}(x) = \sum_{i=1}^n b_i(k) \hat{f}_i^{\text{KDE}}(x)$$
where the weights $b_i(k)$ are derived from $(\Pi^\top \Pi)^{-1}\Pi^\top$. This oracle estimator achieves the optimal minimax rate.

C. The Proposed Estimator (Plug-in with De-biasing)

Since $\Pi$ is unknown, the authors estimate it using the estimated topic matrix $\hat{G}$ from the histogram data.

1. Plug-in: They construct a plug-in estimator by replacing $\Pi$ with an estimate $\hat{\Pi}$ derived from $\hat{G}$.
2. De-biasing: The naive plug-in estimator suffers from significant bias due to the correlation between the estimated weights and the kernel terms. To correct this, the authors replace quadratic terms in the estimator with incomplete U-statistics (excluding diagonal terms where $j=j'$).

The final estimator takes the form:
$$\hat{g}(x) = \hat{G}^\top \hat{G} (\hat{G}^\top T \hat{G})^{-1} \hat{G}^\top S(x) \mathbf{1}_n$$
where $S(x)$ and $T$ are matrices constructed from the data using U-statistic principles to ensure unbiasedness.

4. Key Contributions

1. New Estimator: A novel "Topic-Weighted Kernel Density Estimator" that effectively leverages the induced topic model structure to de-mix nonparametric densities.
2. Minimax Optimality: The paper establishes that the proposed estimator achieves the minimax optimal rate of convergence for the integrated squared error (ISE).
   • The rate is: $O\left( K \left( \frac{K}{nN} \right)^{\frac{2\beta}{2\beta+d}} \right)$.
   • This rate depends on the total sample size $nN$, the number of components $K$, the dimension $d$, and the smoothness $\beta$.
3. Theoretical Analysis of Topic-SCORE: The authors provide a new theoretical analysis of the Topic-SCORE algorithm for the case where the number of topics $K$ grows with the sample size. They prove that Topic-SCORE achieves the optimal rate for estimating the topic matrix even when $K$ is large, which is a result of independent interest.
4. Lower Bound: A matching minimax lower bound is derived, confirming that no other estimator can achieve a faster rate under the same assumptions.

5. Main Results and Convergence Rates

• Convergence Rate: Under the assumption that the base densities belong to the Nikol'ski class with smoothness $\beta$, and provided $K$ grows slowly enough relative to $nN$ (specifically $K = O((nN)^{c^*})$ for some $c^* < 1/4$), the estimator achieves:
  $$\mathbb{E} \left[ \sum_{k=1}^K \int (\hat{g}_k(x) - g_k(x))^2 dx \right] \asymp K \left( \frac{K}{nN} \right)^{\frac{2\beta}{2\beta+d}}$$
• Comparison with Austern et al. (2025):
  • For low smoothness ($\beta \le 1$), both methods perform similarly.
  • For high smoothness ($\beta > 1$), the proposed method is strictly faster. Austern et al.'s method suffers from a suboptimal rate of $O((nN)^{-2/3})$ or worse for $\beta > 1$ due to the inability to choose an optimal bin size $M$ that balances discretization bias and estimation variance. The proposed method avoids this by using the U-statistic de-biasing technique.
• Tuning Parameters:
  • Bin Size ($M$): Set to $M \asymp K \log(nN)$. The method is robust to $M$ as long as it is in this range.
  • Bandwidth ($h$): Selected via cross-validation or set to the optimal order $h \asymp (K/nN)^{1/(2\beta+d)}$.

6. Significance and Impact

• Bridging Discrete and Continuous: The work successfully bridges the gap between discrete topic modeling (text analysis) and continuous nonparametric density estimation, providing a rigorous framework for "continuous topic models."
• LLM Applications: It offers a theoretical foundation for analyzing Large Language Models (LLMs) where "topics" are represented as distributions in high-dimensional embedding spaces rather than discrete word counts.
• Statistical Theory: The derivation of the minimax rate for unmixing nonparametric densities with growing $K$ fills a significant gap in the literature. The use of incomplete U-statistics to de-bias plug-in estimators in high-dimensional settings is a powerful technique that may be applicable to other latent variable problems.
• Computational Efficiency: By leveraging existing efficient topic modeling algorithms (like Topic-SCORE) as a subroutine, the method avoids the computational complexity of variational EM or MCMC methods often used in nonparametric mixture models.

In summary, this paper provides a theoretically optimal solution to the problem of unmixing nonparametric densities, demonstrating that by combining topic modeling with de-biased kernel estimation, one can recover base distributions at the minimax optimal rate, outperforming existing methods especially in high-smoothness regimes.