🤖 machine learning

Who Trains Matters: Federated Learning under Enrollment and Participation Selection Biases

This paper addresses the persistent performance gap in federated learning caused by both enrollment and participation selection biases by formalizing a two-stage selection model and proposing \textsc{FedIPW}, an inverse-probability-weighted aggregation scheme that effectively recovers target-population objectives even when client-level covariates are limited.

Original authors: Gota Morishita

Published 2026-04-30

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Gota Morishita

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to bake the perfect cake for a whole city. To do this, you ask thousands of home bakers to send you a small piece of their batter so you can mix it together and figure out the ideal recipe. This is essentially how Federated Learning (FL) works: instead of gathering all the data in one place, a central server asks many devices (like phones) to train a model locally and send back only the "updates" (the batter pieces).

The problem, as this paper explains, is that who you ask to send batter matters just as much as how you mix it.

The Two-Stage Filter: Who Gets in the Door?

The paper argues that in real life, the bakers you end up hearing from are rarely a perfect cross-section of the whole city. This happens in two distinct stages, like a two-step security check at a concert:

The "Enrollment" Bias (Who gets the invitation?):
First, you have to be eligible to join the project. Maybe you need a specific type of phone, a certain software version, or you just have to click "I Agree" on a consent form. If your phone is old or you live in an area with poor internet, you never even get the invitation. You are filtered out before the game even starts. The paper calls this Enrollment Bias.
- Analogy: Imagine you only invite people who own a red car to your baking club. Even if you ask everyone with a red car to participate, you've already missed everyone with a blue car, a bike, or no vehicle at all. Your "baking club" is already skewed.
The "Participation" Bias (Who actually shows up?):
Second, even among the people who did get invited, not everyone shows up to every meeting. Maybe their battery is dead, their internet is spotty, or it's 3 AM in their time zone. They are enrolled, but they don't participate in that specific round. The paper calls this Participation Bias.
- Analogy: Even if you invited everyone with a red car, maybe only the ones who are awake and have a full tank of gas actually drive to the meeting.

The Problem: Baking the Wrong Cake

Most existing methods try to fix the second problem (who shows up). They say, "Okay, the people who showed up tonight are mostly night-shift workers; let's adjust the recipe to account for that."

But this paper points out a bigger issue: If the people who got invited in the first place (the red car owners) aren't like the rest of the city, fixing the "who shows up" part won't help. You might perfectly adjust for the night-shift workers, but you are still baking a cake based entirely on red-car owners. The final result will taste great to red-car owners but terrible to everyone else.

The paper calls this a "Target-Population Mismatch." The model learns to serve the people who are reachable, not the people it's supposed to serve.

The Solution: A Weighted Scale (FedIPW)

To fix this, the author proposes a new method called FedIPW (Federated Inverse Probability Weighting).

Think of this as using a weighted scale instead of a simple average.

The Old Way (FedAvg): If 10 people send updates, you give each person 1/10th of the weight.
The New Way (FedIPW): You look at who didn't show up and ask, "Why?"
- If a group of people (say, people with Android phones) rarely gets invited because of strict software rules, but they do show up when they are invited, the algorithm gives their updates extra weight.
- If a group (say, people with new iPhones) is invited often but rarely shows up, their updates are also weighted carefully to represent the ones who did show up.

By mathematically "re-weighting" the updates based on the probability of being invited and the probability of showing up, the server can reconstruct what the "average city baker" would have contributed, even if they never actually sent a piece of batter.

What If We Don't Have All the Details? (The "Limited Information" Fix)

Sometimes, the server doesn't know the details of the people who didn't get invited (e.g., it doesn't know how many people in the city have old phones). It only knows the big picture (e.g., "20% of the city uses Android").

In this case, the paper suggests a Calibration trick.

Analogy: Imagine you are baking with a sample of bakers, but you don't know the exact demographics of the whole city. However, you do have a census report that says, "The city is 50% men and 50% women."
If your sample of bakers is 80% men, you can't just ignore the women. Instead, you give the men's updates less weight and the women's updates more weight until your sample looks like the census report (50/50).
This doesn't fix everything perfectly, but it gets you much closer to the right recipe than doing nothing.

The "Bias Floor" Warning

The paper also warns about a "Bias Floor."
Imagine you are trying to hit a bullseye. If you miss the target slightly because your aim is shaky (random error), you can get better with practice. But if your gun is bent (structural error), you will always miss the center, no matter how much you practice.

The paper proves that if you ignore the "Enrollment" stage (the bent gun), you will hit a Bias Floor. No matter how many rounds of training you do, the model will never reach the true best solution for the whole population. It will get stuck in a "good enough" zone that is actually wrong for the people you care about.

Summary

The Issue: Federated learning often fails because the people who join the training (enrollment) and the people who actually participate are not representative of the whole population.
The Fix: Use a two-step correction (FedIPW) that mathematically weighs the updates to account for who was excluded at the start and who dropped out during the process.
The Takeaway: It's not enough to just fix who shows up to the meeting; you have to fix who was invited to the meeting in the first place. If you don't, your model will be biased toward a specific group, no matter how smart the algorithm is.

1. Problem Statement

Federated Learning (FL) aims to train a global model on a target population of distributed clients without centralizing raw data. Standard algorithms (e.g., FedAvg) implicitly assume that the clients participating in each communication round are a representative sample of the target population. However, in practice, this assumption often fails due to selection bias, which the paper identifies as occurring at two distinct stages:

Enrollment Bias: Determined by eligibility rules (device constraints, software requirements, user consent) that dictate which clients are ever reachable for training. This creates a mismatch between the enrolled pool and the target population.
Participation Bias: Determined by dynamic factors (battery, network, local time) that dictate which enrolled clients actually participate in a specific communication round.

The Core Issue: Existing literature primarily addresses round-level participation bias. However, the paper argues that even if participation bias is perfectly corrected, enrollment bias induces a persistent mismatch between the training objective (optimizing for the enrolled subset) and the target-population objective. If the enrolled pool is not representative, the model converges to a suboptimal solution for the intended population.

2. Methodology

The paper formalizes FL under a two-stage selection model and proposes a debiasing framework based on Inverse Probability Weighting (IPW).

A. Two-Stage Selection Framework

The authors model client inclusion as a causal process:

Stage 1 (Enrollment): A client $i$ is enrolled ( $E_i=1$ ) with probability $\pi^{enroll}_i = P(E_i=1 | Z_i)$ , where $Z_i$ are pre-enrollment covariates (e.g., device type, region).
Stage 2 (Participation): Conditional on enrollment, client $i$ participates in round $r$ ( $A_{i,r}=1$ ) with probability $\pi^{part}_{i,r} = P(A_{i,r}=1 | E_i=1, Z_i, X_{i,r})$ , where $X_{i,r}$ are pre-round covariates (e.g., battery, network).

The overall inclusion probability factorizes as:
$p_{i,r} = \pi^{enroll}_i \times \pi^{part}_{i,r}$

B. The Proposed Algorithm: FedIPW

To recover the target-population mean update, the authors propose FedIPW (Federated Inverse Probability Weighting).

Mechanism: Instead of averaging updates equally, the server weights each client's update $\Delta_{i,r}$ by the inverse of their inclusion probability:
$\hat{\bar{\Delta}}_r = \frac{1}{N} \sum_{i: A_{i,r}=1} \frac{\Delta_{i,r}}{\hat{p}_{i,r}}$
Estimation: Since true probabilities are unknown, FedIPW estimates the two components separately:
1. $\hat{\pi}^{enroll}_i$ is estimated using pre-enrollment covariates and enrollment indicators (requiring population-level data).
2. $\hat{\pi}^{part}_{i,r}$ is estimated among enrolled clients using round-level covariates.
Theoretical Guarantee: Under ignorability and positivity assumptions, FedIPW provides an unbiased estimator of the target-population mean update.

C. Limited-Information Extension: Aggregate Calibration

In many real-world scenarios, individual-level covariates for non-enrolled clients are unavailable, making full enrollment IPW impossible.

Solution: The authors introduce an Aggregate-Calibration extension.
Method: It reweights the enrolled sample so that the weighted distribution of pre-enrollment covariates (e.g., region, OS version) matches known aggregate summaries of the target population (e.g., census data).
Role: This acts as a partial correction for enrollment bias when full individual-level data is missing.

3. Key Contributions

Formalization of Two-Stage Selection: The paper distinguishes enrollment bias from participation bias, formalizing them as a sequential causal process. It demonstrates that ignoring enrollment bias leads to a structural mismatch that participation-only corrections cannot fix.
FedIPW Algorithm: Derivation of an inverse-probability-weighted aggregation scheme that recovers the target-population objective under standard assumptions.
Optimization Analysis with Bias Floor: The authors provide an algorithm-agnostic analysis showing that residual weighting errors (due to estimation error or structural misspecification, such as omitting the enrollment stage) induce a non-vanishing bias floor.
- The error bound decomposes into a transient term, a variance term, and a bias floor proportional to $\epsilon_w^2 G^2 / \mu$ , where $\epsilon_w$ is the weighting error and $G$ is gradient heterogeneity.
- This proves that incomplete selection correction (e.g., fixing only participation) results in persistent suboptimality.
Aggregate Calibration: A practical method for limited-information regimes that uses population summaries to partially mitigate enrollment bias.

4. Experimental Results

The authors validate their theory using synthetic federated logistic regression, where the true target-population optimum can be computed accurately.

Objective Mismatch: Naive FedAvg converges to a "selected-client solution" with significantly higher target-population loss compared to the true optimum.
FedIPW Performance: FedIPW successfully recovers the target-population objective, closely tracking the "Oracle" (ideal) performance even as enrollment bias increases.
Participation-Only Failure: Methods that correct only round-level participation improve over FedAvg but fail to reach the optimum as enrollment bias strengthens, confirming the existence of the bias floor.
Aggregate Calibration: In settings where individual non-enrolled data is missing, aggregate calibration significantly reduces the residual error left by participation-only methods, though it is sensitive to noise in the population summaries.

5. Significance and Implications

Paradigm Shift: The paper argues that "who can train at all" (enrollment) is as critical as "who participates in a round" (participation). Current FL debiasing efforts focusing solely on round-level selection are insufficient if the enrolled pool is unrepresentative.
Structural vs. Statistical Error: The analysis distinguishes between statistical estimation errors (which can be reduced with better models) and structural misspecification (e.g., omitting the enrollment stage), which creates an irreducible bias floor.
Practical Guidance: The work provides a roadmap for practitioners:
1. If individual data for non-enrolled users is available, use FedIPW with two-stage weighting.
2. If only aggregate population data is available, use Aggregate Calibration to partially correct enrollment bias.
3. Relying solely on participation correction is theoretically insufficient for achieving population-aligned objectives in heterogeneous environments.

In summary, this paper establishes that achieving a model representative of the target population in FL requires correcting selection bias at both the enrollment and participation stages, offering a rigorous mathematical framework and practical algorithms to achieve this.