💻 computer science

FedSS: Shareability-Aware Privacy-Preserving Federated Learning for Sports Recognition

The paper proposes FedSS, a shareability-aware framework for differentially private federated learning in sports recognition that selectively shares transferable sport semantics while retaining client-specific context locally to mitigate clipping distortion and improve the privacy-utility trade-off.

Original authors: Jiahui Lin, Jie Wu, Ke Sun, Yuanyuan Huang

Published 2026-10-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiahui Lin, Jie Wu, Ke Sun, Yuanyuan Huang

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of sports, the difference between a champion and a contender often lies in the subtle details of movement, posture, and technique. To teach a computer to recognize these details, we need vast amounts of images showing athletes in action. However, these images are rarely gathered in one central place. They are scattered across schools, professional clubs, training centers, and individual gyms, each holding a unique collection of photos. This distribution creates a significant hurdle for artificial intelligence: how can a computer learn from all these images without forcing every organization to hand over their private data? The answer lies in a method called federated learning. Instead of moving the photos to a central server, the computer model travels to the local data. It learns from the images on-site and sends back only a summary of what it has learned, leaving the original photos untouched. This protects privacy, but it introduces a new problem. Even these summaries can sometimes reveal sensitive information about the people in the photos. To fix this, researchers add a layer of mathematical noise to the summaries, a technique known as differential privacy, which scrambles any potential clues about individuals. Yet, this safety measure comes with a cost. The noise can blur the very details the computer needs to learn, making it harder to distinguish between different sports or athletes.

A team of researchers has developed a new approach to solve this specific tension between privacy and accuracy in sports recognition. They call their system FedSS. The core idea behind their work is that not all information learned from a sports photo is equally valuable to share. When a computer looks at a picture of a basketball player, it learns two distinct types of things. First, it learns the universal rules of the sport: the way a player jumps, the angle of a knee during a shot, and the general shape of the ball. These are the "transferable" secrets that help the computer recognize basketball anywhere. Second, it learns the specific details of that particular photo: the color of the team's uniform, the lighting in that specific gym, the background crowd, or the unique camera angle. These details are specific to that one client and do not help the computer understand the sport itself. In traditional privacy-preserving systems, the computer treats all this information as a single package. It adds noise to the whole bundle, which means the valuable, universal rules get blurred along with the useless, specific details. This wastes the limited "space" available for sending information and distorts the most important knowledge.

The researchers at the University of Chinese Academy of Sciences, Kyungil University, and Chengdu University of Information Technology proposed a smarter way to handle this. They built a system that acts like a filter before the information leaves the local computer. Before sending any data, the system analyzes the learned information and separates the universal sports rules from the local, specific details. It then applies the privacy protection only to the universal rules, keeping the local details behind the scenes. This ensures that the limited amount of information allowed to be sent is packed entirely with the most useful knowledge. To make this separation work, they created a module that evaluates the information based on three criteria. It checks how much a piece of information helps identify the sport, how consistent that information is across different teams and locations, and how stable it remains when noise is added. If a piece of information is highly useful for recognizing the sport and stays consistent across different clients, the system marks it for sharing. If it is just a reflection of a specific gym's lighting or a specific team's uniform, the system keeps it local.

The team tested this method using two large collections of sports images, one containing 22 different sports and another with 100. They simulated a realistic environment where many different organizations participated, each with their own unique mix of athletes, uniforms, and camera conditions. They compared their new system against several existing methods that try to protect privacy. The results showed that by filtering out the local noise before sending data, their system was significantly better at recognizing sports. In the most difficult test scenarios, where the data was very different from one client to another, their method improved the accuracy of sports recognition by several percentage points compared to the best previous methods. This improvement was consistent even when the researchers made the privacy protection stronger, which usually makes the task much harder. The system also proved to be more efficient; because it sent less "junk" data, the mathematical noise required for privacy had less of a negative impact on the final result.

The researchers also looked closely at what happened inside the system to understand why it worked. They found that the system successfully kept the computer focused on the actual movements and techniques of the athletes, rather than getting distracted by the background or the uniforms. This led to a model that understood the sport itself, rather than just memorizing the specific photos it had seen. The study confirmed that this approach works well even when the number of participating organizations is very large or when only a small fraction of them participate in any given round of training. The findings suggest that for tasks like sports recognition, where the goal is to understand a specific activity across many different environments, it is crucial to distinguish between what is universal and what is local. By doing so, it is possible to protect the privacy of the athletes and the institutions without sacrificing the ability of the computer to learn effectively. The work demonstrates that privacy and utility do not have to be enemies; with the right strategy, they can work together to create a smarter, more secure system for analyzing the world of sports.

Technical Summary: FedSS – Shareability-Aware Privacy-Preserving Federated Learning for Sports Recognition

1. Problem Statement

Federated Learning (FL) offers a paradigm for collaborative sports image recognition without centralizing raw data, addressing privacy concerns regarding athlete information and institutional data ownership. However, standard FL approaches face two critical challenges in sports applications:

Visual Heterogeneity: Sports data distributed across different clients (schools, clubs, venues) exhibit significant visual variations in athlete appearance, uniforms, backgrounds, illumination, and viewpoints, even when the sport category is identical. This creates a conflict between transferable sport semantics (e.g., motion patterns, body configurations) and client-specific visual context.
Privacy-Utility Trade-off: To ensure rigorous privacy, Differentially Private Federated Learning (DP-FL) applies gradient clipping and Gaussian noise. Conventional DP-FL treats all model updates uniformly. Consequently, low-shareability, client-specific contextual variations consume a portion of the limited clipping range ( $L_2$ norm), causing unnecessary distortion of the high-value, transferable sport semantics. This leads to significant utility degradation under privacy constraints.

Existing methods often improve optimization stability or sparsification but fail to explicitly distinguish between globally transferable knowledge and local context before applying privacy mechanisms.

2. Methodology: The FedSS Framework

The authors propose FedSS (Shareability-Aware Privacy-Preserving Federated Learning), a framework designed to decouple transferable sport semantics from client-specific context before privacy perturbation.

Core Components:

Sport Semantic Shareability Module (SSSM): This module evaluates network stages (e.g., in a ResNet-18 backbone) to estimate a "shareability score" ( $q_{i,b}^t \in [0,1]$ $q_{i, b}^{t} \in [0, 1]$ ) for each stage. The score is derived from three criteria:
1. Sport Discriminativeness: Measures how strongly a stage contributes to correct sport recognition using gradient-activation relevance.
2. Cross-Client Invariance: Measures the consistency of the current update with an Exponential Moving Average (EMA) of previously shared private updates, ensuring alignment with global knowledge.
3. Privacy Robustness: Evaluates the stability of a stage's gradient under local perturbation (using Gradient-Aware Minimization, GAM) and its signal strength relative to expected Gaussian noise.
Semantic-Context Decoupling: Using the estimated shareability scores, the candidate update ( $\Delta_i^t$ $Δ_{i}^{t}$ ) is softly decomposed into:
- Transferable Component ( $\Delta_{i,sport}^t$ ): Weighted by shareability scores, this component is privatized (clipped and noise-added) and uploaded.
- Context Component ( $\Delta_{i,context}^t$ ): The residual is retained locally in a Local Context Memory ( $r_i$ ) to preserve client-specific adaptations (e.g., specific uniforms or venues) without exposing them to the server.
Local Optimization: Clients utilize Gradient-Aware Minimization (GAM) to optimize local models, characterizing first-order flatness to improve robustness against noise.
Server Aggregation: The server aggregates only the privatized sport-semantic updates to update the global model and refreshes the EMA references for the next round.

Privacy Guarantees:
Theoretical analysis (Proposition 1) confirms that because the shareability coefficients and local residuals are never transmitted, and the final released update is bounded and privatized using standard client-level DP mechanisms, FedSS maintains the same $(\epsilon, \delta)$ -differential privacy guarantees as standard DP-FL without requiring additional privacy accounting.

3. Key Contributions

Framework Proposal: Introduction of FedSS, which explicitly distinguishes globally transferable sport semantics from client-specific visual context in heterogeneous federated settings.
Shareability Module: Development of the SSSM, which adaptively evaluates stage-wise shareability based on discriminativeness, invariance, and robustness, guiding the selective sharing of knowledge.
Theoretical Analysis: Proof that the stage-wise soft filtering is non-expansive (Proposition 2), meaning it cannot increase clipping distortion, and derivation of error bounds showing reduced semantic suppression and context leakage (Propositions 3 & 4).
Comprehensive Evaluation: Extensive experiments on sport22 and sport100 datasets under various heterogeneity settings (label skew, visual context shift, and combined) and privacy budgets.

4. Experimental Results

Experiments were conducted on ResNet-18 backbones across different client scales (100 and 500 clients) and privacy budgets ( $\epsilon \in [4, 12]$ ).

Accuracy: FedSS consistently outperformed representative DP-FL baselines (including DP-FedAvg, DP-FedSAM, and DPSIR-Fed). Under the challenging Mixed Heterogeneity (M-Het) setting, FedSS improved Top-1 accuracy by 5.78% on sport22 and 5.71% on sport100 compared to the strongest baseline (DPSIR-Fed).
Privacy-Utility Trade-off: FedSS demonstrated superior performance across all privacy budgets, with particularly significant gains under strong privacy constraints ( $\epsilon=4$ ).
Mechanism Analysis:
- Clipping Distortion: FedSS significantly reduced clipping distortion (ClipDist) by filtering out low-shareability context before clipping.
- Semantic Consistency: The method achieved higher Sport Semantic Consistency (SCS) and lower Class-Conditional Client-Context Predictability (C-CCP), indicating that the shared model learned more about the sport itself and less about client-specific artifacts.
Ablation Studies: Removing the SSSM caused the largest performance drop, confirming it as the core driver of improvement. The local context memory and GAM optimization provided complementary gains.

5. Significance and Claims

The paper claims that FedSS effectively mitigates the utility degradation typically associated with privacy-preserving federated learning in sports recognition. By recognizing that not all learned information is equally valuable for global sharing, FedSS optimizes the limited "clipping budget" to preserve high-value, transferable sport semantics while retaining low-value, client-specific context locally.

The authors assert that this shareability-aware approach provides a robust solution for heterogeneous sports data, offering:

Improved recognition accuracy under strict privacy guarantees.
Reduced distortion of useful knowledge during the privacy mechanism.
Stronger cross-client semantic consistency in the global model.

The work suggests that future research could extend this shareability-aware paradigm to video data, multimodal inputs, and more granular adaptive estimation, but the current contribution is firmly grounded in static image recognition benchmarks.