🤖 machine learning

Test-Time Collective Action: Proxy-Based Perturbations for Correcting Algorithmic Harms

This paper proposes Test-Time Collective Action (TTCA), a framework enabling a coordinated group of users to correct algorithmic disparities in black-box systems by pooling query access to train a proxy model and apply universal perturbations at inference time, thereby improving subgroup accuracy and fairness metrics without requiring platform-side retraining.

Original authors: Meghana Bhange, Ulrich Aïvodji, Elliot Creager

Published 2026-05-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Meghana Bhange, Ulrich Aïvodji, Elliot Creager

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The Broken Vending Machine

Imagine a giant, high-tech vending machine (the AI platform) that sells snacks to everyone. It works great for most people. However, for a specific group of customers—let's say, people wearing red hats—the machine keeps giving them the wrong snack or refusing to sell anything at all.

Usually, if a vending machine is broken, the only way to fix it is to call the company that owns it (the platform provider) and ask them to send a repair crew. But what if the company is too busy, doesn't care, or says, "We can't fix it right now"? The people in red hats are stuck. They have no way to make the machine work for them without the company's help.

The Old Way: The "Data Donation" Approach

In the past, researchers suggested that if enough people in red hats sent the company their photos and data, the company could "retrain" the machine to work better.

The Catch: This only works if the company agrees to stop the machine, take the new data, and rebuild it. In the real world, companies rarely do this on demand. The users have no control over the machine's internal gears.

The New Idea: "Test-Time Collective Action" (TTCA)

This paper proposes a clever workaround. Instead of trying to fix the machine's inside, the group of people in red hats learns how to trick the machine's outside just enough to get the right snack.

Think of it like a group of friends trying to figure out the secret code to a locked door. They don't have the blueprints (the AI's internal code), but they can knock on the door and listen to the sound it makes.

Here is how the process works, step-by-step:

1. The "Mock Machine" (Proxy Extraction)

The group pools their resources. Instead of each person knocking on the door one by one, they send a batch of questions (queries) to the real machine.

The Analogy: Imagine 100 people each asking the vending machine, "What happens if I put in a coin?" The machine answers. The group collects all these answers and builds a fake vending machine (a "proxy") in their garage. This fake machine isn't perfect, but it mimics the real one's behavior closely enough to learn from.

2. Finding the "Magic Wrench" (Optimizing Perturbations)

Now, the group experiments on their fake machine. They try to find a tiny, almost invisible adjustment (a "perturbation") that, when added to a user's input, forces the machine to give the correct answer.

The Analogy: They discover that if you tap the vending machine's glass just right before inserting your coin, it dispenses the right snack. They figure out exactly how hard to tap and where to tap.
The "Universal" Trick: They don't need a different tap for every single person. They find one specific tap that works for everyone in the red hat group who wants a specific snack. This is called a "universal perturbation."

3. The "Magic Tap" in Action (Test-Time Application)

Now, any person in the red hat group who wants to use the real vending machine doesn't need to ask the company for help. They simply apply their "magic tap" (the perturbation) to their request before sending it to the real machine.

The Result: The real machine, which was previously confused, now sees the request clearly and gives the correct snack. The group fixed the problem without ever touching the machine's internal gears or asking the owner for permission.

Why is this better than acting alone?

The paper compares two scenarios:

Acting Alone: One person tries to fix their own problem by knocking on the door thousands of times, trying different angles until they get lucky. This is expensive and slow.
Acting Together (Collective): The group shares the cost. They knock on the door a few thousand times once to build their "fake machine" and find the "magic tap." Then, that single "tap" works for thousands of people forever.

The Paper's Key Findings:

Small Groups Work: You don't need a massive army. Even a small group (as few as 5–10 people) pooling their resources can create a fix that works better than if they tried to fight the machine alone.
It's Cheaper: It takes far fewer "knocks" (API queries) for the group to succeed than for individuals to succeed on their own.
It Works on Different Machines: Even if the group builds their "fake machine" using a simple model, the "magic tap" they find still works on the complex, high-tech real machine.
It Fixes Fairness: By helping the under-served group, the system becomes fairer for everyone, not just the lucky majority.

The Bottom Line

This paper shows that when a computer system treats a specific group of people unfairly, that group doesn't have to wait for the company to fix it. By working together, they can build a "shadow model" of the system, figure out a tiny tweak that corrects the errors, and apply that tweak themselves. It turns a group of powerless users into a collective force that can correct the machine's mistakes from the outside.

Technical Summary: Test-Time Collective Action (TTCA)

Problem Statement

Machine learning systems frequently exhibit performance disparities across user subgroups, resulting in "quality-of-service" (QoS) harms where specific groups experience systematically lower accuracy than others. While provider-centric fairness interventions (pre-, in-, and post-processing) exist, they rely on the platform's willingness and ability to retrain models. In scenarios where remediation is unavailable, delayed, or the platform refuses to act, affected users lack external levers to correct these disparities.

Existing "Algorithmic Collective Action" (ACA) frameworks allow users to steer algorithms by contributing data, but these mechanisms typically require the platform to retrain on the collective's modified data—a condition users rarely control. Furthermore, data-poisoning style interventions can be circumvented if providers wait for robust model updates or train on pre-attack data. Consequently, there is a need for a user-side intervention that operates at test time on a frozen black-box model without requiring platform cooperation or access to the training loop.

Methodology: Test-Time Collective Action (TTCA)

The authors propose TTCA, a framework enabling a coordinated group of users to correct algorithmic harms by pooling query access to a black-box API. The mechanism operates in two phases:

Phase 1: Collective Proxy Extraction and Perturbation Optimization

Query Pooling: A collective of $n$ users, each with a query budget $B$ , pools their resources to query the platform model $f_{plat}$ on a public, unlabeled image pool $D_{pub}$ . This generates a dataset $D_{proxy}$ of input-output pairs $(x, f_{plat}(x))$ .
Proxy Training: The collective trains a surrogate model (proxy) $f_{proxy}$ on $D_{proxy}$ using the KnockoffNets method. To improve transferability, an ensemble of $K=5$ proxies is trained with independent random seeds.
Perturbation Optimization: For each class $y$ in the under-served subgroup $G$ , the collective optimizes a per-class universal perturbation $\delta_y$ . The objective is to minimize the cross-entropy loss of the proxy ensemble for inputs belonging to class $y$ , subject to an $\ell_\infty$ budget ( $\|\delta_y\|_\infty \le \epsilon$ ). The optimization targets the ground-truth class, effectively "pushing" misclassified inputs toward the correct decision boundary. A total-variation regularizer and Gaussian blur are applied to bias the perturbation toward low-frequency components, enhancing black-box transferability.

Phase 2: Test-Time Application

At inference time, members of the under-served subgroup apply the pre-computed perturbation $\delta_y$ to their inputs $x$ before submission. If a user's input belongs to class $y \in G$ , they submit $x' = \text{clip}(x + \delta_y, 0, 1)$ ; otherwise, they submit $x$ unchanged. This process requires no further interaction with the platform beyond the initial query budget used to build the proxy.

Key Contributions

Test-Time Corrective Mechanism: TTCA introduces a framework where users can correct disparities on a frozen, black-box model without retraining the platform. It repurposes evasion attack logic (universal adversarial perturbations) for a corrective, rather than malicious, purpose.
Proxy-Based Efficiency: By extracting a collective proxy and optimizing a reusable universal perturbation, the method amortizes the cost of intervention across the entire subgroup.
Cross-Architecture Transfer: The mechanism demonstrates that a smaller proxy model (e.g., ResNet-18) can effectively generate perturbations that transfer to larger, unknown platform architectures (e.g., ResNet-50, RegNet-Y).
Cost Efficiency Analysis: The paper provides a rigorous analysis showing that pooling queries is significantly cheaper than individual per-image attacks, even with small collective sizes.

Experimental Results

The authors evaluate TTCA on CIFAR-10, CIFAR-100, and FairFace (for demographic fairness) under both balanced and deliberately unbalanced platform training regimes.

Subgroup Accuracy: TTCA significantly improves the accuracy of the under-served subgroup (defined as the bottom 10% of groups) at very low query budgets. For instance, on CIFAR-100, a collective of just 5–6 users pooling queries can achieve subgroup accuracy comparable to or exceeding individual attackers, while using orders of magnitude fewer total queries.
Query Budget Efficiency:
- Individual Baseline: Individual users attacking alone using SimBA (a per-image black-box attack) require up to 308× more total platform queries to reach comparable subgroup accuracy.
- Break-even Point: A collective of as few as 5 to 10 members is sufficient to make pooling more cost-effective than individual intervention.
- Ceiling Trade-off: While individual attacks can achieve higher maximum accuracy (e.g., 97.9% on CIFAR-100) by optimizing per-image, they do so at a massive aggregate cost (~~2.3 million queries for 1,000 users). The collective approach saturates at a lower accuracy ceiling (~~83%) but achieves this with a fixed, reusable cost (~23,000 queries), making it far more efficient for large groups.
Fairness Metrics: The intervention improves standard group-fairness metrics, including Worst-Group Accuracy (WGA), Equal-Opportunity Gap (EOD), and Disparate Impact (DI), without degrading performance for other groups.
Proxy Fidelity: High-fidelity proxies are not strictly necessary; once a proxy is "faithful enough" (e.g., 60–90% fidelity), additional queries yield diminishing returns in subgroup accuracy.

Significance and Claims

The paper positions TTCA as a corrective intervention for users facing QoS harms when platform-side remediation is unavailable. Its significance lies in:

Agency: It provides users with an external lever to regain agency over algorithmic systems without needing access to model internals or training data.
Scalability: It demonstrates that coordinated, small-scale user action can effectively mitigate systemic disparities through resource pooling.
Complementarity: It acts as a complementary approach to existing ACA frameworks, specifically addressing the "test-time" gap where models are already deployed and immutable.

The authors note that this approach is most effective when the underlying task is broadly useful but unevenly performed. They acknowledge limitations: if the task itself is harmful (e.g., automated gender recognition causing representational harms), TTCA's corrective framing may be insufficient, and users may need to resort to refusal or contestation rather than optimization. Additionally, the method assumes a static black-box model; platform-side defenses (e.g., randomized smoothing) could potentially mitigate the effectiveness of these perturbations, though such defenses may also degrade overall platform accuracy.