📊 statistics

Design-based edge-level causal inference with machine learning assisted covariate adjustment

This paper proposes a design-based framework for edge-level causal inference in directed networks under dyadic interference, introducing Horvitz-Thompson estimators with improved variance bounds and a novel three-fold cross-fitting procedure to enable efficient, machine learning-assisted covariate adjustment while addressing the unique dependence structures of edge outcomes.

Original authors: Haoyang Yu, Yilin Li, Lu Deng, Yong Wang, Xin Lu, Hanzhong Liu

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Haoyang Yu, Yilin Li, Lu Deng, Yong Wang, Xin Lu, Hanzhong Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out if a new rule in a game changes how players interact. In most standard experiments, you look at individual players: "Did Player A win more points?" But in this paper, the authors are looking at something different: the interactions themselves. They are studying the "edges" (the connections) between players, like a message sent from Player A to Player B.

Here is a simple breakdown of their work, using some creative analogies.

1. The Problem: The "Butterfly Effect" of Connections

Usually, scientists assume that what happens to one person doesn't affect another (like two people in separate rooms). But in social networks, people are connected. If you change the treatment for Person A, it might change how they talk to Person B.

The authors focus on directed edges (one-way connections, like an email or a text). The outcome isn't just about Person A or Person B; it's about the relationship between them.

The Analogy: Imagine a crowded dance floor. If you change the music for one dancer (the sender), it changes how they dance with their partner (the receiver). But if you also change the music for the partner, the dance changes again. The "dance" (the outcome) depends on both dancers' music settings.
The Challenge: Because these dances are linked (if A dances with B, and B dances with C, they share a person), standard math tools break. It's like trying to count the total weight of a pile of sand where every grain is glued to its neighbors; you can't just add them up one by one.

2. The Solution: The "Three-Part Puzzle" (Sample Splitting)

To fix the math, the authors needed a way to make sure their predictions were fair. Usually, in machine learning, you split data into two piles: one to train a model, one to test it.

The Problem with Two Piles: In a network, if you put Person A in the "Training" pile and Person B in the "Testing" pile, but they are connected, the model is cheating. It's using information from the test group to guess the test group's outcome because they are linked.
The Fix (Three-Fold Splitting): The authors invented a three-way split. Imagine dividing the dance floor into three colored zones: Red, Blue, and Green.
- To predict the dance between a Red dancer and a Blue dancer, you train your model using data from the Green zone (and the other parts of Red/Blue that don't touch them).
- This ensures that the model predicting a specific interaction has never "seen" the people involved in that specific interaction before. It's like hiring a judge who has never met the two people arguing in court to ensure a fair verdict.

3. The Tool: The "Smart Assistant" (Machine Learning)

The authors wanted to make their estimates more precise. They used Machine Learning as a "smart assistant" to guess what the interactions would look like based on other clues (like how often these people talked before).

The Risk: Sometimes, a smart assistant guesses wrong, or guesses in a way that accidentally makes the final answer less accurate than if you hadn't used it at all.
The Safety Net (Calibration): The authors added a "calibration step." Think of this as a safety check. If the smart assistant's guess is too wild or unhelpful, the system automatically reverts to a simpler, safer method. This guarantees that using the fancy AI will never make the results worse than the basic method; it will either make them better or leave them the same.

4. The Results: Sharper, Faster Answers

The paper proves mathematically that their new method works and tested it in two ways:

Simulations: They created fake networks on computers. They found that their method was much more efficient (gave a clearer answer with less "noise") than old methods. The "smart assistant" (Machine Learning) was especially good when the relationships were complicated and non-linear.
Real-World Test: They applied this to a real experiment on WeChat (a massive Chinese social media app). They looked at how a change in the recommendation algorithm affected how much users shared content with each other.
- The Outcome: The new method detected a small but significant increase in sharing activity that the old, "dumber" methods might have missed or been too unsure about.

Summary

In short, this paper is about how to measure the effect of a change on relationships rather than just on people.

They realized standard math fails because relationships are tangled.
They created a three-way split to keep the math honest.
They added a safety check so that using advanced AI never hurts the results.
They proved it works on fake data and successfully used it to find subtle effects in a real-world social network.

The core message is: When studying connections, you need a special kind of math that respects the web of relationships, and a safety net to ensure your computer models help rather than hinder.

Technical Summary: Design-based Edge-level Causal Inference with Machine Learning Assisted Covariate Adjustment

1. Problem Statement

This paper addresses the challenge of conducting design-based causal inference for edge-level outcomes in directed networks under dyadic interference. Unlike traditional node-level settings where outcomes are defined on individual units, edge-level outcomes (e.g., communication intensity, trade flows, disease transmission) are defined on directed pairs $(i, j)$ and depend on the joint treatment assignments of both the sender ( $Z_i$ ) and the receiver ( $Z_j$ ).

This setting introduces a complex dependence structure that invalidates standard estimation and inference procedures developed for node-level data. Specifically:

Dependence: Edge observations sharing common endpoints are inherently dependent, violating the independence assumptions often used in standard sample splitting.
Bias in Node Aggregation: Aggregating edge outcomes to the node level (e.g., summing outgoing interactions) often discards the specific sender-receiver treatment configuration information required for unbiased estimation of edge-level causal effects.
Variance Estimation: Classical variance estimators relying on Cauchy-Schwarz inequalities are overly conservative because they treat all cross-products of potential outcomes as unobservable, ignoring identifiable components unique to the edge-level structure.

2. Methodology

A. Framework and Estimators
The authors adopt a design-based framework where potential outcomes and network structure are fixed, and randomness arises solely from the treatment assignment vector $Z$ .

Dyadic Interference Assumption: The potential outcome $Y_{i \to j}$ depends only on $(Z_i, Z_j)$ .
Causal Estimands: A general class of edge-level causal effects is defined as $\tau = \sum_{\zeta} \omega(\zeta)\theta(\zeta)$ , where $\theta(\zeta)$ is the average potential outcome under treatment configuration $\zeta \in \{(0,0), (0,1), (1,0), (1,1)\}$ . This allows for specific effects like sender effects, receiver effects, and total effects.
Horvitz–Thompson (HT) Estimator: The authors construct HT estimators for $\theta(\zeta)$ and $\tau$ . They establish that these estimators are unbiased and derive their asymptotic normality under mild regularity conditions (bounded outcomes, sufficient network connectivity, and specific growth rates for network density $\rho_n$ ).

B. Variance Estimation
To avoid the over-conservatism of classical approaches, the paper develops refined variance estimators by exploiting the identifiability of certain cross-products of potential outcomes.

Identifiability Rule: A cross-product $Y_{i \to j}(z_1, z_2)Y_{i' \to j'}(z_3, z_4)$ is identifiable if it does not impose conflicting treatment assignments on the same unit.
Refined Bounds: Instead of diagonalizing the entire variance matrix, the method retains compatible cross-products (e.g., reciprocal edges $Y_{i \to j}Y_{j \to i}$ $Y_{i \to j} Y_{j \to i}$ with consistent treatments) and only bounds the remaining incompatible ones. This yields two estimators:
- $\widehat{var}_{U1}$ : A finite-sample upper bound (conservative).
- $\widehat{var}_{U2}$ : An asymptotically conservative target that is less conservative than $\widehat{var}_{U1}$ when the average network degree is large.

C. Covariate Adjustment and Three-Fold Sample Splitting
To improve efficiency, the paper incorporates auxiliary covariates using a machine learning-assisted Augmented Inverse Probability Weighting (AIPW) approach.

The Challenge: Standard two-fold sample splitting fails in edge-level settings because an edge may connect a node in fold A to a node in fold B. Training a model on one fold would require excluding data from both endpoints to ensure conditional independence, potentially leaving no data for training.
The Solution: A three-fold sample splitting and cross-fitting scheme is introduced. Nodes are partitioned into three disjoint sets ( $S_1, S_2, S_3$ ). Edges are categorized based on the fold membership of their endpoints (within-fold or between-fold). Models are trained on specific combinations of folds that ensure the training data for an edge $(i, j)$ does not share endpoints with the edge itself.
Stability Condition: The asymptotic normality of the adjusted estimator relies on a stability condition requiring the fitted models to converge to the oracle function at a sufficient rate relative to the network size and density.
Calibration: A calibration step is introduced for non-linear machine learning models to guarantee that the adjusted estimator does not suffer an asymptotic efficiency loss relative to the unadjusted estimator (the "do-no-harm" property).

3. Key Contributions

Theoretical Foundation: Establishment of convergence rates and central limit theorems for Horvitz–Thompson estimators of general edge-level causal effects under weaker network assumptions than existing literature.
Refined Variance Estimation: Development of variance estimators that leverage identifiable edge-level dependence structures, yielding substantially less conservative bounds than classical Cauchy-Schwarz-based approaches.
Three-Fold Cross-Fitting: Introduction of a tailored three-fold sample splitting procedure that restores the conditional independence required for unbiased covariate adjustment in the presence of edge-level dependence, enabling the use of flexible machine learning methods.
High-Dimensional and Non-Parametric Validity: Verification of the stability condition for both linear models (allowing covariate dimension $d = o(n)$ ) and non-parametric machine learning methods, alongside a calibration step to ensure efficiency gains.

4. Results

Simulations: Extensive simulations across varying network sizes, sparsity levels, and data-generating processes (linear and non-linear) demonstrate that:
- The proposed estimators are unbiased.
- Covariate-adjusted estimators (Linear Regression, B-Splines, Random Forest) achieve substantial variance reduction (e.g., ~10% reduction compared to linear adjustment in non-linear settings) compared to unadjusted estimators.
- The second variance estimator ( $\widehat{var}_{U2}$ ) provides confidence intervals with coverage probabilities closer to the nominal 95% level and shorter lengths compared to the first estimator.
- The calibration step effectively prevents variance inflation in non-linear models.
Real-Data Application: The methods were applied to a large-scale A/B test on WeChat Channels involving billions of directed edges.
- The study evaluated the effect of an updated fine-ranking formula on sender-to-receiver interaction intensity.
- The Random Forest-adjusted estimator with the second variance estimator detected a statistically significant positive effect (p-value = 0.049) with a smaller standard error (0.63%) compared to the unadjusted estimator (0.72%), demonstrating practical efficiency gains.

5. Significance and Claims

The paper claims to provide a rigorous, design-based framework for causal inference in directed networks where outcomes are defined on edges. Its primary significance lies in:

Bridging the Gap: Addressing the methodological gap where standard node-level causal inference tools fail to directly extend to edge-level outcomes due to complex dependence structures.
Efficiency: Demonstrating that incorporating covariates via machine learning, when combined with the proposed three-fold splitting and calibration, can significantly improve estimation precision without sacrificing validity.
Robustness: Offering valid inference that does not rely on correctly specifying outcome models, as the design-based framework treats randomization as the sole source of uncertainty.
Practical Applicability: Validating the approach on real-world, large-scale platform data, showing that these theoretical advances can be deployed in industrial settings to evaluate interventions on network interactions.

The authors acknowledge that the framework assumes dyadic interference (dependence only on endpoints) and that extensions to higher-order interference (e.g., neighbors of neighbors) via exposure mappings remain an area for future work. They also note the increased data and computational requirements of edge-level analysis compared to node-level analysis.