💻 computer science

Toward Individual Fairness Without Centralized Data: Selective Counterfactual Consistency for Vertical Federated Learning

This paper proposes SCC-VFL, a server-centric framework for vertical federated learning that enforces individual-level counterfactual fairness by combining differentially private feature role discovery, masked counterfactual generation, and a consistency loss to significantly reduce decision flip rates while maintaining predictive accuracy and privacy.

Original authors: Dawood Wasif, Chandan K. Reddy, Terrence J. Moore, Jin-Hee Cho

Published 2026-05-12

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Dawood Wasif, Chandan K. Reddy, Terrence J. Moore, Jin-Hee Cho

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are applying for a loan. Your financial life is like a puzzle, but the pieces are scattered across different buildings. Bank A has your account history, your employer has your pay stubs, and a credit bureau has your debt records. None of them want to hand over their private files to a central boss because of privacy laws. So, they use a system called Vertical Federated Learning (VFL). Think of it like a group of chefs in separate kitchens who each have different ingredients. They send only small, cooked samples of their dishes to a head chef (the server) who mixes them to taste the final meal, without ever seeing the raw ingredients in the other kitchens.

The problem is: Is the final decision fair?

If you are 25 years old, will you get the loan? If you were 26, would you get it? If the answer changes just because of your age (a protected trait), that's unfair. But it's even trickier here because the "age" piece of the puzzle might be in one kitchen, while the "job stability" piece is in another. The system might accidentally use your age to guess your job stability and then use that to deny your loan, even if age shouldn't matter.

The authors of this paper, SCC-VFL, propose a new way to fix this. They call it Selective Counterfactual Consistency. Here is how it works, using simple analogies:

1. The "What-If" Test (Counterfactuals)

Imagine you are a time traveler. You go back and change one thing about yourself: your age. You ask, "If I were 26 instead of 25, but everything else about my life stayed exactly the same, would I still get the loan?"

Fair: If the answer is "Yes, I still get the loan," the system is stable.
Unfair: If the answer is "No, I get rejected," the system is unstable and potentially biased.

The goal is to make sure the answer stays the same, even if you change your age.

2. The Problem with "Blind" Fixes

Previous attempts to fix this were like trying to edit a photo by blurring the whole picture. They would try to remove all traces of age from the data. But this is too harsh. Sometimes, age does matter legally or logically (e.g., a 21-year-old can't rent a car, but a 25-year-old can). We need to be smart about what we change.

3. The SCC-VFL Solution: The "Smart Editor"

The authors built a system that acts like a Smart Editor with three specific tools:

Tool A: The "Privacy Mask" (Finding the Roles)

Before editing, the system needs to know which features are which.

Non-Descendants (The "Fixed" Anchors): These are things that cannot change just because your age changed. For example, your loan amount or how long you've lived in your house. These must stay exactly the same in the "What-If" scenario.
Mediators (The "Allowed" Movers): These are things that can naturally change with age. For example, your job tenure or credit history might look different for a 26-year-old than a 25-year-old. The system is allowed to update these to match the new age.
Proxies (The "Forbidden" Leaks): These are sneaky features that secretly tell the system your age without you saying it (like a specific zip code or a demographic code). The system must block these from influencing the decision.

How do they find these without seeing the raw data? They use a Privacy Sketch. Imagine the server asks the party holding the age data to send a "blurred" version of the age list (like a foggy photo) that protects individual privacy but shows the general pattern. Using this foggy photo, the system figures out which features are "Anchors," "Movers," or "Leaks" without ever seeing the real names or ages.

Tool B: The "Masked Generator" (The Safe Edit)

Once the roles are assigned, the system creates a "What-If" version of your application.

It locks the Anchors (Non-Descendants) so they don't budge.
It updates the Movers (Mediators) to look realistic for the new age (e.g., slightly longer job tenure).
It guards the Leaks (Proxies) so they don't accidentally whisper the old age to the decision-maker.

This is like a tailor who is allowed to adjust the sleeves of a suit (Mediators) to fit a new size, but is strictly forbidden from changing the fabric pattern (Anchors) or adding a hidden tag that reveals the original owner (Proxies).

Tool C: The "Stability Check" (Server-Side Enforcement)

Finally, the server looks at the original decision and the "What-If" decision.

If the decision flips (e.g., "Approved" becomes "Rejected") just because of the allowed changes, the system gets a penalty.
It forces the model to learn a way of deciding that is stable. It teaches the model: "If the only things that changed were the things we allowed to change, the result should stay the same."

The Results: A Fairer, Safer System

The authors tested this on three real-world scenarios: Credit (loans), Healthcare (heart disease risk), and Criminal Justice (recidivism risk).

The Outcome: Their system (SCC-VFL) was much better at keeping decisions stable. In some cases, it reduced "decision flips" (where a person gets a different answer just because of a protected trait) by 98% compared to other methods.
No Trade-Off: Usually, making a system fairer makes it less accurate (like a security guard who stops everyone to be safe). But SCC-VFL kept the accuracy high while making the decisions fairer.
Privacy: It also made it harder for hackers to guess your private traits (like age or race) by looking at the data the system shared.

Summary

Think of SCC-VFL as a Fairness Filter for a team of secret chefs. It ensures that when they combine their secret ingredients to make a decision, the final taste doesn't change arbitrarily just because one secret ingredient (like your age) is slightly different. It does this by:

Blurring the sensitive info to protect privacy.
Sorting features into "Fixed," "Allowed to Change," and "Forbidden."
Editing the "Allowed" parts realistically.
Punishing the system if the final decision changes unfairly.

The result is a system that respects privacy, keeps data distributed, and ensures that your loan or medical decision depends on your actual qualifications, not on a hidden bias against your age, race, or gender.

Technical Summary: Selective Counterfactual Consistency for Vertical Federated Learning (SCC-VFL)

1. Problem Statement

The paper addresses the challenge of ensuring individual fairness in Vertical Federated Learning (VFL) settings where data features are distributed across multiple institutions (parties), and sensitive attributes (e.g., race, age, gender) may be held privately by a single party or an enclave.

In VFL, parties hold disjoint feature subsets for overlapping entities. While federated learning allows collaborative model training without centralizing raw data, it complicates fairness enforcement. Existing fairness approaches often rely on group-level metrics (e.g., demographic parity, equalized odds) or assume full access to features and causal graphs, which are unavailable in VFL. Furthermore, naive counterfactual interventions in VFL are unstable because:

Partial Visibility: No single party observes all features, making it difficult to determine which features are permissible mediators versus impermissible proxies of a sensitive attribute.
Privacy Constraints: Sensitive attributes are often private, preventing direct intervention during training.
Leakage: Shared representations or updates can inadvertently leak sensitive information or allow proxies to transmit sensitive influence.

The authors focus on individual-level counterfactual stability: ensuring that an individual's prediction remains unchanged under a hypothetical intervention on their protected attribute, provided that non-descendant features are held fixed and only policy-permitted mediators are allowed to vary.

2. Methodology: SCC-VFL

The authors propose SCC-VFL (Selective Counterfactual Consistency for Vertical Federated Learning), a server-centric framework that operationalizes individual fairness through three integrated components:

A. Graph-Free, Differentially Private (DP) Mask Discovery

Since causal graphs are rarely available in multi-party settings, SCC-VFL uses a graph-free, DP procedure to partition each party's features into three sets:

Non-descendants ( $N$ ): Features that should remain fixed under a sensitive attribute intervention.
Mediators ( $M$ ): Features that are permissible to change (policy-permitted pathways).
Proxies ( $P$ ): A subset of mediators that act as impermissible pathways for sensitive influence and must be suppressed.

Mechanism:

A trusted holder (or enclave) generates a DP sketch ( $z_s$ ) of the sensitive attribute $s$ by clipping and adding Gaussian noise to contingency tables. This sketch satisfies $(\epsilon, \delta)$ -DP but is not applied to the full training pipeline.
Parties use $z_s$ as an auxiliary covariate to test if their local features are statistically responsive to $s$ .
The system computes two metrics for each feature:
- Risk Gain ( $\Delta_j$ ): Improvement in predicting the feature when $z_s$ is included.
- Hilbert-Schmidt Independence Criterion (HSIC): Residual dependence on $z_s$ after conditioning on other local features.
Features with low HSIC are assigned to $N$ ; those with high HSIC are candidates for $M$ . Within $M$ , features with high risk gain (strong sensitivity to $s$ but low task relevance) are flagged as proxies $P$ .

B. Masked Counterfactual Generation

Each party trains a local conditional generator (a cVAE) to create counterfactual inputs.

Identity Preservation: Non-descendants ( $N$ ) are copied exactly ( $x^{cf}_N = x_N$ ).
Mediator Editing: Mediators ( $M$ ) are edited to lie on the support of the conditional distribution $p(x_M | x_N, c, z_{s'})$ , where $z_{s'}$ is the target sensitive embedding.
Leakage Suppression: Proxies ( $P$ ) pass through unchanged but are subject to an adversarial loss. An adversary attempts to recover $z_s$ from the counterfactual proxy features; the generator minimizes this success rate to suppress residual sensitive signals.

C. Server-Side Selective Counterfactual Consistency (SCC) Loss

The server aggregates representations from original and masked counterfactual inputs to form fused representations $z$ and $z^{cf}$ .

Consistency Penalty: The server minimizes a loss term $\mathcal{L}_{SCC}$ that penalizes the difference between predictions $f_\theta(z)$ and $f_\theta(z^{cf})$ .
Objective: This enforces that the model's decision does not flip arbitrarily when only policy-permitted mediators are edited. The total objective combines task utility, SCC consistency, and validity constraints (identity, support, leakage).

3. Key Contributions

Problem Formulation: Defined Selective Counterfactual Consistency (SCC) for VFL, targeting per-instance stability under policy-specified interventions rather than group parity.
DP Mask Discovery: Introduced a graph-free, differentially private mechanism to identify candidate non-descendants, mediators, and proxies without requiring a pre-defined causal graph or raw sensitive data access.
Masked Generation with Leakage Control: Designed party-local generators that edit only mediators while enforcing identity on non-descendants and suppressing proxy leakage via adversarial training.
Server-Centric Enforcement: Implemented a consistency loss at the server that penalizes impermissible prediction changes, validated across diverse domains.

4. Experimental Results

The authors evaluated SCC-VFL on three real-world datasets: German Credit (Banking), UCI Heart Disease (Healthcare), and COMPAS Cox (Criminal Justice), under both IID and Non-IID (distribution shift) splits.

Fairness Metrics: SCC-VFL significantly reduced the Flip Rate (FR) (the fraction of samples changing prediction under intervention) by up to 98% compared to strong baselines (e.g., Adv-NoMask, Uniform-CF, Policy-blind Mask). It also achieved the lowest Selective Consistency Gap (SCG).
- Example: On German Credit (Non-IID), SCC-VFL reduced FR to 0.12% compared to 6.24–8.22% for baselines.
Utility: The method maintained or improved predictive accuracy and LogLoss, demonstrating favorable utility-fairness trade-offs.
Privacy and Robustness:
- Attribute Inference Attacks (AIA): SCC-VFL reduced attack success rates by 4–5 percentage points compared to baselines.
- Subspace-PGD Attacks: Under targeted perturbations of mediator coordinates, SCC-VFL showed the lowest success rates (e.g., 9.3% at $\epsilon=0.20$ vs. 21.7% for Adv-NoMask), indicating flattened decision boundaries along sensitive directions.
Ablation Studies: Removing any component (mask discovery, generator, or consistency loss) degraded performance, confirming the necessity of the full pipeline for stability and leakage control.

5. Significance and Claims

The paper claims that SCC-VFL provides a practically auditable fairness target for VFL deployments where data is siloed and sensitive attributes are private.

Individual Accountability: Unlike group-level metrics that can mask individual inconsistencies, SCC-VFL ensures that specific individuals receive stable decisions under counterfactual changes to protected attributes, aligning with legal requirements against discrimination (e.g., age discrimination in lending).
Policy Alignment: The framework enforces a declared policy regarding which causal pathways are permissible. It does not claim to discover ground-truth causality but rather operationalizes a governance-defined specification ( $N/M/P$ ) into a learning algorithm.
Robustness to Shift: The method remains effective under Non-IID settings and distribution shifts, where baseline methods often suffer from stability degradation.
Privacy Scope: The authors explicitly state that the formal DP guarantee applies only to the sketch used for mask discovery, not to the full training pipeline (representations, gradients, or model parameters). The privacy benefits observed in experiments are empirical reductions in leakage under specific threat models, not formal end-to-end DP guarantees.

The work positions individual-level counterfactual stability as a viable complement to group fairness, offering a mechanism for stakeholders to audit, contest, and verify that algorithmic decisions are not arbitrarily driven by protected attributes in distributed learning environments.