Client-Cooperative Split Learning

The paper presents CliCooper, a multi-client cooperative Split Learning framework that safeguards data privacy through differential privacy and label obfuscation while ensuring training integrity and model provenance via dynamic chained watermarking, effectively mitigating privacy and ownership attacks without compromising model accuracy.

The Gist

Imagine you have a secret recipe for the world's best chocolate cake (your data). You want to hire a team of chefs to help you perfect this recipe, but you have two big problems:

1. You are broke: You don't have a giant industrial kitchen (computing power) to bake the cake yourself.
2. You don't trust the chefs: You are worried that if you give them the recipe, they might steal it, figure out your secret ingredients, or claim they invented the cake themselves.

This is exactly the problem the paper CLICOOPER solves. It's a new way to train Artificial Intelligence (AI) models without anyone having to show their raw data or trust a single "boss" server.

Here is how it works, broken down into simple analogies:

1. The Setup: The "Potluck" Kitchen

In traditional AI training, you send your secret recipe to a giant, trusted cloud server (the "Boss"). The Boss does all the work.

• The Problem: What if you don't have a Boss? What if you have to hire a bunch of independent chefs (Trainer Clients) who each have a tiny bit of oven space?
• The CLICOOPER Solution: Instead of one big kitchen, you create a relay race.
  • You (The Data Owner): You have the ingredients but no oven. You do the prep work.
  • The Chefs (Trainers): They have ovens but no ingredients. They pass the dough down the line.
  • The Verifier: A neutral referee who watches the race to make sure everyone did their job and pays them fairly.

2. The Secret Sauce: Hiding the Ingredients (Privacy)

You can't just hand the chefs the raw ingredients, or they might figure out your secret recipe. CLICOOPER uses two magic tricks:

• Trick A: The "Fake Label" Menu (Label Expansion)
  Imagine your cake has 10 secret flavors (True Labels). Instead of telling the chefs "This is Chocolate," you give them a menu with 20 fake names (Pseudo-Labels).

  • Real Chocolate becomes "Flavor A1" and "Flavor A2."
  • Real Vanilla becomes "Flavor B1" and "Flavor B2."
  • Why it works: The chefs see 20 flavors, but they don't know which ones are actually the same. They can't guess your original 10 flavors. Only you hold the "decoder ring" to translate their work back to the real flavors.

• Trick B: The "Static Noise" Filter (Differential Privacy)
  When you pass the dough to the first chef, you sprinkle a little bit of "static noise" on it.

  • It's like adding a tiny bit of flour dust that makes the dough look slightly blurry.
  • Why it works: If a chef tries to look at the dough to guess what the original ingredients were (an "Inversion Attack"), the noise makes it impossible. They see a blurry mess, not a clear picture.

3. The Proof of Work: The "Chain of Custody" (Watermarking)

Now, imagine a chef tries to cheat. They say, "I baked this layer of the cake!" but they actually just grabbed a pre-made cake from a store and claimed they made it. How do you know they actually did the work?

CLICOOPER uses a Chained Watermark.

• Think of it like a secret handshake that changes every time.
• Chef #1 finishes their layer and passes it to Chef #2.
• Chef #2 looks at the exact dough Chef #1 passed them. Based on that specific dough, Chef #2 generates a unique, invisible "stamp" (watermark) and puts it on their own layer.
• Chef #3 does the same: they look at Chef #2's layer, generate a new stamp, and add it.
• The Result: The final cake has a chain of invisible stamps. If you try to take a layer out and put it on a different cake, the stamps won't match. If a chef tries to skip the race and use a pre-made layer, they can't generate the correct stamp because they didn't see the previous layer.

4. The Referee: The Verifier

At the end of the race, the Verifier (the referee) checks the cake.

1. Did it taste good? They check if the cake is accurate (did the model learn?).
2. Did everyone do their part? They check the chain of invisible stamps. If the stamps match the chain, they know every chef actually baked their part.
3. Payment: If everything checks out, the Verifier pays the chefs. If a chef cheated, they get nothing.

Why is this a big deal?

• No Single Boss: You don't need a massive, expensive server. You can use many small, cheap devices (like phones or edge computers).
• Total Privacy: The chefs never see your raw data, your real labels, or your secret mapping. They only see blurry, fake-named data.
• No Theft: Even if someone steals the final cake, they can't use it without your "decoder ring" (the secret mapping). It's useless to them.
• Fairness: You can prove exactly who baked which part of the cake, so everyone gets paid fairly.

In short: CLICOOPER turns AI training into a secure, cooperative relay race where you can hire a team of strangers to build a model for you, without ever having to show them your secrets or worry about them stealing the credit.

Technical Summary

Here is a detailed technical summary of the paper "Client-Cooperative Split Learning" (CLICOOPER) by Deng et al.

1. Problem Statement

The paper addresses the limitations of traditional Split Learning (SL) in resource-constrained, serverless environments. While standard SL allows data owners to train models without sharing raw data by offloading computation to a trusted server, this paradigm assumes a single, fully trusted server with sufficient compute power.

In reality, edge devices often possess fragmented but surplus compute power. This motivates a multi-client, serverless cooperative setting where one client provides data (Data Client, $C$) and multiple other clients provide compute (Trainer Clients, $T$). However, this setting introduces three critical challenges in a partially trusted environment:

• RQ1 (Data Privacy): How can the Data Client protect raw inputs and true labels from curious Trainer Clients who observe intermediate activations?
• RQ2 (Layer Ownership): How can Trainer Clients prove their specific contributions to the model to claim fair compensation, preventing "free-riding" (using pre-trained models)?
• RQ3 (Unauthorized Use): How can the final model be protected against unauthorized extraction or repurposing by external adversaries?

Existing SL frameworks fail to simultaneously address label privacy, training traceability, and copyright protection in this decentralized, multi-client context.

2. Methodology: The CLICOOPER Framework

CLICOOPER is a multi-client cooperative SL framework designed for heterogeneous, partially trusted environments. It integrates three core mechanisms to address the research questions:

A. Secret-Mapping Label Expansion (Addressing RQ1 & RQ3)

To hide label semantics and quantities without degrading model utility:

• Mechanism: The Data Client ($C$) maps each true label $Y_i$ to a set of $g_i$ pseudo-labels ($Y^*_{ij}$) using a secret one-to-many mapping $G_Y$.
• Data Augmentation: The dataset is augmented to align with the expanded label space, ensuring class balance.
• Effect: Trainer Clients only see pseudo-labels. Without the secret inverse mapping $G_Y^{-1}$, the model's outputs are operationally useless to unauthorized parties, effectively binding model utility to authorization.

B. Differential Privacy (DP) Guarded Activations (Addressing RQ1)

To prevent feature inversion and clustering attacks on intermediate activations:

• Mechanism: Before uploading activations, $C$ applies $\ell_1$-clipping to bound sensitivity and injects calibrated Laplace noise based on a privacy budget $\epsilon$.
• Theoretical Guarantee: The paper proves that under this mechanism, the likelihood ratio of inferring that two activations belong to the same class is bounded by $e^{2\epsilon}$, significantly limiting linkage attacks.
• Workflow: $C$ performs the expansion and noise injection once, caching the DP-protected activations ($M^D_C$) for the training pipeline.

C. Dynamic Chained Watermarking (Addressing RQ2)

To ensure verifiable training integrity and ownership attribution:

• Mechanism: After the model converges (Epoch $N$), a watermarking phase (Epoch $N+1$) occurs. Each Trainer $T_i$ embeds a unique watermark $\Lambda_{Ti}$ into their model segment.
• Chaining: The watermark is cryptographically chained. The watermark for $T_i$ is deterministically derived from a hash of the previous trainer's output activation ($M_{T_{i-1}}$), the trainer's identity, and a secret nonce.
  • $H_1 = H(M^D_C, 1, \mu_1, ID_{T1})$
  • $H_i = H(M_{T_{i-1}}, i, \mu_i, ID_{Ti})$
• Verification: A trusted Verifier ($V$) reconstructs the expected watermark chain. If a trainer skips training or uses a pre-trained model, the activation input will differ, causing the derived watermark to mismatch, thus detecting "free-riding."

3. Key Contributions

1. Novel Framework: Introduced CLICOOPER, the first framework tailored for serverless, multi-client cooperative SL that balances privacy, ownership, and utility.
2. Privacy-Preserving Design: Combined label expansion and DP noise to suppress label inference and feature inversion attacks while maintaining model accuracy.
3. Cryptographic Provenance: Developed a chained watermarking scheme that links training stages across distributed clients, enabling auditable ownership claims and deterring unauthorized model reuse.
4. Comprehensive Evaluation: Validated the system across diverse datasets (MNIST, CIFAR-10/100, AG News) and architectures (CNNs, Transformers), demonstrating robustness against internal and external attacks.

4. Experimental Results

The authors conducted extensive experiments comparing CLICOOPER against baselines and various attack scenarios:

• Model Accuracy: CLICOOPER preserves baseline accuracy, with some configurations showing up to a 2% improvement (attributed to noise acting as a regularizer).
• Defense Against Clustering Attacks:
  • Without protection, clustering attacks recover label groups with 100% accuracy.
  • With CLICOOPER (DP + Label Expansion), the success rate drops to 0% for image datasets (CIFAR-10/100) and is significantly reduced for text/MNIST.
• Defense Against Inversion Attacks:
  • Unprotected activations allow reconstruction with high similarity (SSIM ~0.50).
  • With DP noise ($\epsilon=2.0$), reconstruction similarity drops to 0.03, rendering the recovered images unrecognizable.
• Defense Against Model Extraction:
  • External adversaries attempting to train surrogate models via API queries achieve only ~1% accuracy (equivalent to random guessing) due to the pseudo-label obfuscation.
• Overhead:
  • Watermarking: Embedding and verification take only milliseconds (e.g., ~8-13ms), negligible compared to training time.
  • Communication: Latency scales linearly with segment boundaries but remains low (e.g., <1.5s total for CIFAR-100).

5. Significance

CLICOOPER represents a significant advancement in decentralized AI training:

• Economic Viability: It enables a "compute marketplace" where edge devices can monetize idle resources for model training without compromising data privacy.
• Trustless Collaboration: By removing the need for a single trusted server and introducing cryptographic proofs of work, it solves the "trust gap" in multi-party collaboration.
• Holistic Security: Unlike previous works that focus solely on privacy or solely on copyright, CLICOOPER provides a unified solution that protects data owners, ensures fair compensation for trainers, and secures the final model against theft.

In summary, CLICOOPER transforms Split Learning from a server-centric protocol into a robust, serverless, and economically viable framework for collaborative AI in partially trusted environments.