Technical Summary: Multivariate Species Sampling Models

1. Problem Statement and Motivation

In Bayesian nonparametrics (BNP), the assumption of infinite exchangeability is foundational, linking exchangeable sequences to random probability measures via de Finetti's representation theorem. However, in many applied settings involving distinct but related data sources (e.g., meta-analysis, multi-center studies, or topic modeling), strict exchangeability is overly restrictive as it fails to account for heterogeneity across groups. Conversely, assuming independence between groups precludes the sharing of information, which is often a primary objective.

The natural compromise is partial exchangeability, where data are exchangeable within groups but not necessarily across them. While numerous models for partially exchangeable data have been proposed—collectively known as dependent nonparametric priors (e.g., hierarchical, nested, and additive processes)—they lack a unifying framework. Consequently, key questions regarding their underlying learning mechanisms and dependence structures remain unresolved. Specifically, the literature relies heavily on pairwise correlation measures (e.g., $\text{Cor}[P_j(A), P_k(A)]$) to quantify dependence, yet it is often obscure why this measure is independent of the set $A$, how it translates to observable dependence, and whether it reliably characterizes independence or full exchangeability.

2. Methodology: Multivariate Species Sampling Processes (mSSPs)

The authors introduce Multivariate Species Sampling Processes (mSSPs) as a general class of nonparametric priors designed to unify and extend existing dependent processes.

2.1 Definition and Structure

An mSSP is defined as a vector of random probability measures $(P_1, \dots, P_J)$ such that:
$$P_j \stackrel{a.s.}{=} \sum_{h \ge 1} \pi_{j,h} \delta_{\theta_h} + \left(1 - \sum_{h \ge 1} \pi_{j,h}\right) P_0$$
where:

• $P_0$ is a non-atomic deterministic base measure.
• $\theta = (\theta_h)_{h \ge 1}$ are i.i.d. atoms drawn from $P_0$.
• $\pi = (\pi_1, \dots, \pi_J)$ is a random array of sub-probability sequences (weights).
• The dependence among the $P_j$'s is induced entirely by the dependence structure of the weight array $\pi$, while the atoms $\theta$ remain independent of the weights.

This formulation generalizes univariate Species Sampling Processes (SSPs) by allowing the weights to be shared or distinct across populations. The authors provide an alternative representation (Proposition 2) that explicitly decomposes the weights into shared components (atoms common to multiple populations with positive probability) and idiosyncratic components (atoms unique to a specific population).

2.2 Regular mSSPs (rmSSPs)

A notable subclass, termed regular mSSPs (rmSSPs), is defined by imposing an independence condition on the relative frequencies of non-shared species. Specifically, for any pair of populations, the normalized weights of the idiosyncratic components must be independent. The authors argue that this subclass is particularly relevant because it encompasses almost all currently used dependent nonparametric priors in BNP, including:

• Hierarchical processes (e.g., Hierarchical Dirichlet Process, Hierarchical Pitman-Yor).
• Nested processes (e.g., Nested Dirichlet Process).
• Additive processes (e.g., Additive Dirichlet Process).
• Completely random vectors and various stick-breaking constructions.

3. Key Contributions and Theoretical Results

3.1 Characterization via Partially Exchangeable Partition Probability Function (pEPPF)

The authors characterize mSSPs through their partially exchangeable partition probability function (pEPPF). They establish a fundamental link (Theorem 5) showing that any pEPPF, when paired with an independent non-atomic base measure, uniquely identifies an mSSP. This implies that the random partition structure of any partially exchangeable array can be studied via the pEPPF of an mSSP, making mSSPs a natural framework for analyzing discrete structures in dependent vectors.

3.2 Dependence Structure and Correlation

A central contribution is the rigorous analysis of the dependence structure, specifically addressing the role of correlation:

• Tie-Driven Dependence: The authors prove (Proposition 4) that the correlation between random measures $P_j(A)$ and $P_k(A)$ is determined exclusively by the probabilities of ties (shared species) within and across groups.
  $$\text{Cor}[P_j(A), P_k(A)] = \frac{P(X_{j,1} = X_{k,1})}{\sqrt{P(X_{j,1} = X_{j,2}) P(X_{k,1} = X_{k,2})}}$$
• Independence of Set $A$: This result explains why correlation is often independent of the set $A$ in existing models: it depends only on the weights (tie probabilities), not the locations.
• Extreme Cases:
  • Correlation = 1: Implies full exchangeability (the measures are almost surely identical).
  • Correlation = 0: For rmSSPs, zero correlation implies independence between the measures (Theorem 2). This characterization does not hold for general mSSPs, highlighting the distinct role of the regular subclass.
• Higher Moments: The paper derives expressions for marginal and mixed moments of mSSPs in terms of observable quantities (e.g., the random number of unique species), extending results previously known only for normalized completely random measures.

3.3 Predictive Structure (mgCRP)

The authors introduce the multivariate generalized Chinese restaurant process (mgCRP) as the predictive mechanism for mSSPs. Unlike standard extensions that treat groups as independent franchises, the mgCRP allocates new observations based on the group of the incoming customer and the group membership of existing customers. The predictive probabilities are expressed as ratios of pEPPFs. For regular mSSPs, the authors show how to construct augmented representations (using latent variables) to make these ratios computationally tractable, facilitating the design of Gibbs samplers for models like HSSP, NSSP, and +SSP.

4. Empirical Results

The paper validates the theoretical framework through a multi-armed bandit experiment aimed at maximizing species discovery across multiple populations.

• Real Data (South American Trees): Using a dataset of tree species across four regions, the authors compared six rmSSP models (Independent, Additive, and Hierarchical versions of Dirichlet and Pitman-Yor processes).
  • Calibration: Models were calibrated to match prior mean and variance of tie probabilities to ensure fair comparison.
  • Findings: All rmSSP models outperformed a uniform random baseline. Models borrowing information (Hierarchical and Additive) generally outperformed independent models, except for the Additive Dirichlet Process (+DP), which underperformed due to restrictive assumptions about shared species frequencies. Pitman-Yor based models consistently outperformed Dirichlet Process based models due to the flexibility of the discount parameter.
• Synthetic Data: A simulation with low probabilities of ties across populations (a "worst-case" scenario for borrowing information) showed that hierarchical and additive rmSSPs performed on par with independent models and close to an oracle. This suggests that borrowing information does not degrade performance even when it is not strictly necessary.

5. Significance and Claims

The paper claims to provide a unifying theoretical foundation for dependent nonparametric priors. Its significance lies in:

1. Unification: Demonstrating that most existing dependent processes are subclasses of mSSPs (specifically rmSSPs).
2. Mechanism Insight: Revealing that information borrowing is entirely determined by shared ties, offering a principled explanation for correlation structures observed in the literature.
3. Constructive Tool: Providing a modular recipe for developing new models by combining EPPFs, allowing for the design of models with alternative probabilistic symmetries.
4. Theoretical Clarity: Resolving open questions regarding the interpretation of correlation and the conditions under which zero correlation implies independence.

The authors maintain a modest stance, noting that while they provide a framework for fair comparison and model calibration, a systematic empirical comparison of all competing models across diverse settings is a direction for future work. They also identify the need for a standalone framework for pEPPFs decoupled from partially exchangeable arrays as a future research avenue.