Inference in high-dimensional logistic regression under tensor network dependence

1. Problem Statement

The paper addresses the challenge of statistical inference (specifically constructing confidence intervals and performing hypothesis tests) for high-dimensional logistic regression models where observations are dependent.

• Context: Traditional high-dimensional inference methods (e.g., debiased Lasso) assume independent and identically distributed (i.i.d.) samples. However, in many real-world scenarios (social networks, spatial data, peer effects), observations are correlated.
• Limitation of Existing Work: Previous literature on dependent data has largely focused on:
  1. Pairwise interactions (e.g., Ising models).
  2. Consistent estimation only, without providing valid inference (confidence intervals or p-values).
  3. Low-dimensional covariates.
• The Gap: There is a lack of theoretical guarantees for inference in high-dimensional logistic regression when dependence is induced by general higher-order structures (hypergraphs/tensor networks) rather than just pairwise graphs.

2. Methodology

The authors propose a two-step procedure to achieve valid inference for low-dimensional linear and quadratic functionals of the regression parameter $\theta$ in a high-dimensional setting ($d \to \infty$ as $n \to \infty$).

Step 1: Regularized Maximum Pseudo-Likelihood Estimation

Due to the computational intractability of the normalizing constant in Markov Random Fields (MRFs), the authors use a Maximum Pseudo-Likelihood Estimator (MPLE).

• Strong Independent Set: To handle dependence, the authors partition the vertices of the underlying hypergraph $G_n$ into a "strong independent set" $S$. Within this set, conditional on the rest of the network, the observations are independent.
• Splitting: The set $S$ is split into two disjoint subsets, $S_1$ and $S_2$.
• Estimation: The initial estimator $\tilde{\theta}$ is obtained by minimizing the $\ell_1$-regularized negative log-pseudo-likelihood on the subset $S_1$:
  $$\tilde{\theta} = \arg\min_b \left\{ L_{S_1}(b) + \lambda_n \|b\|_1 \right\}$$
  where $L_{S_1}$ is the log-pseudo-likelihood conditioned on the rest of the network.

Step 2: Bias Correction (Debiasing)

The $\ell_1$-regularized estimator $\tilde{\theta}$ is biased and cannot be used directly for inference. The authors construct a bias-corrected estimator for a linear functional $c^\top \theta$:
$$\widehat{c^\top \theta} = c^\top \tilde{\theta} + \frac{1}{|S_2|} \sum_{i \in S_2} w_i (y_i - f(\tilde{v}_i)) \hat{u}^\top X_i$$

• Key Components:
  • $f(\cdot)$ is the logistic link function.
  • $w_i$ are data-dependent weights (derived as constant $w_i=2$ in this specific logistic setting).
  • $\hat{u}$ is a projection vector constructed to "de-bias" the estimator. It is chosen to minimize a quadratic form subject to constraints that ensure the bias term vanishes asymptotically.
• Extension to Quadratic Functionals: The method is extended to quadratic functionals $Q = \theta^\top M \theta$ (e.g., heritability estimation) by linearizing the bias term and applying a similar projection-based correction.

3. Key Contributions

1. Generalization to Tensor Networks: Unlike prior work restricted to pairwise Ising models, this framework accommodates general hypergraph-induced dependencies (higher-order peer effects).
2. First Inference Guarantees for High-Dimensional Tensor MRFs: This is the first work to establish asymptotic normality and valid confidence intervals for low-dimensional functionals in high-dimensional logistic regression under general network dependence.
3. Projection-Based Inference: The authors develop a novel construction for the projection vector $\hat{u}$ that accounts for the specific structure of tensor network dependence, proving that the resulting estimator is asymptotically normal.
4. Relaxed Assumptions: The method does not require bounded individual probabilities (a common assumption in previous literature), allowing probabilities to approach 0 or 1, which is realistic in strong network effects.

4. Theoretical Results

The paper provides rigorous non-asymptotic error bounds and asymptotic guarantees under standard high-dimensional assumptions (sparsity $s$, sub-Gaussian covariates, bounded degree of the hypergraph).

• Consistency (Proposition 2.3): The initial MPLE $\tilde{\theta}$ achieves the optimal convergence rate:
  $$\|\tilde{\theta} - \theta\|_2 = O\left(\sqrt{\frac{s \log d}{n}}\right)$$
• Asymptotic Normality (Theorem 2.1): Under sparsity conditions $s = o(\sqrt{n}/\sqrt{\log n} \log d)$, the bias-corrected estimator satisfies:
  $$\frac{\widehat{c^\top \theta} - c^\top \theta}{\sqrt{\hat{V}}} \xrightarrow{d} N(0, 1)$$
  where $\hat{V}$ is a consistent estimator of the asymptotic variance.
• Confidence Intervals & Hypothesis Testing: The authors construct valid $(1-\alpha)$ confidence intervals and demonstrate that the proposed tests control Type-I error and achieve desired power.
• Simultaneous Inference: The framework supports multiple hypothesis testing with control over Family-Wise Error Rate (FWER) and False Discovery Rate (FDR).

5. Simulation Results

The authors validate their theory through three simulation studies:

1. Varying Network Strength: On a 2D lattice, they show that ignoring network dependence (using standard i.i.d. methods) leads to severe under-coverage (e.g., 22% coverage for a 95% CI). Their proposed method maintains coverage near 98%.
2. Varying Network Degree: On random regular graphs with increasing degrees, the proposed method consistently achieves the target coverage, while the baseline fails.
3. Varying Dimension ($d$): As the dimension of the covariates increases, the proposed method remains robust, whereas the baseline method's performance degrades significantly.

6. Significance and Future Directions

• Significance: This work bridges a critical gap between high-dimensional statistics and network science. It enables researchers to perform rigorous statistical inference (e.g., identifying significant peer effects or genetic markers) in complex, dependent data structures that were previously intractable for inference.
• Future Directions:
  • Extending the framework to dense graphs (where strong independent sets are small).
  • Applying the method to causal inference under network interference.
  • Generalizing to non-binary responses and other link functions in generalized linear models.

In summary, the paper provides a robust, theoretically grounded, and computationally feasible framework for statistical inference in high-dimensional logistic regression when data exhibits complex, higher-order dependencies.