Separate Exchangeability as Modeling Principle in Bayesian Nonparametrics

Technical Summary: Separate Exchangeability as a Modeling Principle in Bayesian Nonparametrics

Problem Statement
The paper addresses a gap in Bayesian nonparametric (BNP) inference regarding the treatment of data structured as rectangular arrays (matrices) involving two distinct types of experimental units (e.g., rows and columns). While exchangeability (invariance to all permutations) and partial exchangeability (invariance to permutations within specific groups) are well-established principles, the authors argue that separate exchangeability—invariance to independent permutations of row and column indices—is frequently underutilized in BNP literature despite being the natural modeling principle for such data structures.

The core issue arises when standard BNP models, particularly those based on dependent discrete random measures (like the Dependent Dirichlet Process, DDP), induce partial exchangeability rather than separate exchangeability. In scenarios where the same experimental units (e.g., specific genes, proteins, or OTUs) are observed across different blocks (e.g., subjects, time points, or conditions), partial exchangeability imposes a restrictive symmetry: it allows borrowing of information regarding the distribution of clusters across groups but fails to allow borrowing regarding the specific realization of clusters. For instance, under partial exchangeability, observing that Unit A and Unit B are clustered together in Group 1 does not increase the probability that Unit A and Unit B are clustered together in Group 2, unless the probability of A clustering with any other unit also increases. This ignores the structural identity of the units across the matrix.

Methodology
The authors propose a framework for constructing tractable BNP models that explicitly enforce separate exchangeability. They introduce two primary classes of models:

1. Separately Exchangeable Nested Partitions:

   • Concept: The authors generalize nested partition models (such as the Nested Dirichlet Process, NDP) to respect separate exchangeability. Instead of sharing only the prior law of partitions across groups (as in NDP), the proposed model shares the realization of the partition within clusters of groups.
   • Construction: The model first clusters the columns (groups) using an Exchangeable Partition Probability Function (EPPF). Within each column cluster, the rows (experimental units) are partitioned. Crucially, all columns within a specific cluster share the exact same row partition realization.
   • Implementation: They propose a "Separately Exchangeable NDP-CAM" (Nested Dirichlet Process with Common Atoms). This model uses latent indicators to define column clusters and nested row partitions. It is shown to satisfy the Aldous-Hoover representation theorem for separately exchangeable arrays.
   • Generalization: The framework allows replacing the standard Dirichlet Process (DP) priors with more flexible Gibbs-type priors (e.g., Pitman-Yor Process, Mixture of Finite Mixtures) for both the column partition and the row partition, maintaining computational tractability via generalized Chinese Restaurant Processes (gCRP).

2. Separately Exchangeable Nonparametric Regression:

   • Concept: The authors adapt the Dependent Dirichlet Process (DDP) for regression problems involving two sets of experimental units (e.g., proteins and subjects).
   • Construction: They modify the ANOVA DDP by introducing additive effects in the atoms of the random measure. Specifically, the atoms are defined as linear predictors containing terms specific to the row unit, terms specific to the column unit, and a global term.
   • Formulation: The model assumes $y_{i,j} \sim F_{(i,j)}$, where the random measure $F_{(i,j)}$ has atoms centered at $\beta_h + \xi_i + \eta_j$. Here, $\xi_i$ and $\eta_j$ are independent random effects for rows and columns, respectively. This structure ensures that the joint law is invariant to separate permutations of $i$ and $j$.
   • Flexibility: Similar to the partition models, the priors for the row and column effects can be chosen from Gibbs-type processes to allow for flexible clustering of the regression coefficients.

Key Contributions

• Theoretical Clarification: The paper rephrases existing ad hoc arguments against certain dependent random partition models (e.g., Lee et al., 2013; Page et al., 2022) as consequences of violating separate exchangeability. It provides a formal probabilistic justification for why partial exchangeability is inappropriate when the same experimental units appear across multiple groups.
• Model Construction: It provides the first systematic derivation of separately exchangeable counterparts to widely used partially exchangeable BNP models (specifically NDP and DDP).
• Tractability: The authors demonstrate that these models remain analytically and computationally tractable. They derive the implied Exchangeable Partition Probability Functions (EPPF) and Partial EPPFs (pEPPF), enabling the use of standard Markov Chain Monte Carlo (MCMC) algorithms (specifically partially collapsed Gibbs sampling).
• Borrowing of Strength: The paper clarifies the nature of information borrowing: separate exchangeability allows learning about the specific co-clustering of units in one group to inform the co-clustering in another group, a feature absent in standard partial exchangeability.

Results and Applications
The authors illustrate the methodology using two real-world datasets:

1. Microbiome Data (Diet Swap Study):

   • Data: Frequencies of 119 Operational Taxonomic Units (OTUs) across 38 subjects.
   • Application: The model clusters subjects based on their microbiome profiles while simultaneously clustering OTUs within those subject clusters.
   • Outcome: The model successfully identified subject clusters corresponding to geographic regions (US vs. rural Africa). Crucially, unlike partial exchangeability models, it provided meaningful inference on the co-clustering of specific OTUs within subject clusters, respecting the identity of the OTUs across subjects.

2. Protein Expression Data (Ataxia Study):

   • Data: Abundance of ~3,990 proteins across 32 subjects (16 controls, 16 patients) measured at various ages.
   • Application: A nonparametric regression model was used to identify proteins with differential expression trajectories over age between cases and controls.
   • Outcome: The model estimated protein-specific slope differences ($\gamma_i$) and ranked proteins by the magnitude of these differences. The approach successfully identified top-ranked proteins associated with the disease, demonstrating the utility of separate exchangeability in handling complex, multi-dimensional experimental designs with missing data.

Significance and Claims
The paper argues that separate exchangeability is a superior modeling principle for matrix-structured data where experimental units maintain their identity across different conditions. The authors claim that:

1. Fidelity to Design: Separate exchangeability more faithfully represents the experimental setup than partial exchangeability, which often discards structural information about the design matrix.
2. Necessity for Specific Inference: Certain inferential goals, such as determining if specific units (e.g., OTU 1 and OTU 2) are clustered together in multiple groups, cannot be coherently stated or performed without separate exchangeability.
3. Practical Feasibility: The proposed models are not merely theoretical constructs but are computationally feasible and offer flexible alternatives to existing BNP tools, preserving the ability to perform inference in practice while respecting the specific symmetries of the data.

The authors conclude that careful consideration of the experimental setup often necessitates moving beyond "omnibus" exchangeability or standard partial exchangeability to more specific symmetry assumptions like separate exchangeability.