Hybrid Hierarchical Federated Learning over 5G/NextG Wireless Networking

This paper proposes Hybrid Hierarchical Federated Learning (HHFL), a novel framework that leverages 5G/NextG CoMP capabilities to allow clients in overlapping regions to simultaneously communicate with multiple edge servers, thereby enhancing inter-ES knowledge sharing, mitigating model divergence, and achieving up to twice the convergence speed of traditional HFL, especially under non-IID data conditions.

The Gist

The Big Picture: Teaching a Class with a Twist

Imagine you are trying to teach a massive class of students (the Clients) how to recognize different types of animals. You are the Cloud Server (the Principal), and you have three Edge Servers (the Head Teachers) stationed in different parts of the school building.

In the old way of doing things (Traditional HFL), every student is assigned to exactly one Head Teacher.

1. The Principal sends the "current best lesson plan" to all Head Teachers.
2. Each Head Teacher sends it to their specific students.
3. Students study alone, take notes, and send their notes back to their Head Teacher.
4. The Head Teachers combine the notes from their own students and send a summary back to the Principal.

The Problem: Imagine the Head Teachers are in different wings of the school. The students in the "Cat Wing" only see cats, and the students in the "Dog Wing" only see dogs. Because they never talk to each other, the "Cat Wing" teacher thinks the world is 100% cats, and the "Dog Wing" teacher thinks it's 100% dogs. They are working in silos, and it takes a very long time for the Principal to figure out that the world actually has both cats and dogs.

The New Idea: The "Hybrid" Classroom (HHFL)

The paper proposes a new system called Hybrid Hierarchical Federated Learning (HHFL). This system uses a modern technology called CoMP (Coordinated Multi-Point), which is like giving students a superpower: The ability to talk to multiple teachers at once.

Here is how the new system works:

1. The "Overlapping Zone" Students

In a modern school (5G/NextG networks), the coverage areas of the Head Teachers overlap. Some students sit right on the border between the "Cat Wing" and the "Dog Wing."

• Old Way: We forced these border students to pick just one teacher.
• New Way (HHFL): These border students get lesson plans from both the Cat Teacher and the Dog Teacher simultaneously.

2. The "Knowledge Bridge"

This is the magic part.

• The border student takes the "Cat lesson" and the "Dog lesson," mixes them together (averages them), and studies.
• Because they studied both, they learn that the world has both animals.
• When they finish studying, they send their notes back to both teachers.

The Metaphor: Think of these border students as diplomats or bridges. They carry the "Cat knowledge" into the "Dog Wing" and the "Dog knowledge" into the "Cat Wing." They stop the teachers from getting stuck in their own little bubbles.

Why is this better?

1. Faster Learning (Convergence)
In the old system, the teachers had to wait for the Principal to tell them, "Hey, you're missing half the picture!" This happens slowly.
In the new system, the diplomats (border students) are constantly swapping information. The teachers realize the truth much faster. The paper shows that when the data is messy (some teachers only have cats, some only have dogs), this new system learns twice as fast.

2. Less Wasted Effort
Because the teachers learn faster, they don't have to spend as much time studying the wrong things. Even though the border students are sending notes to two teachers (which uses a tiny bit more energy), the total time the whole school spends learning is much lower.

The "Traffic Jam" Analogy

Imagine the school is a city with traffic.

• Traditional HFL: Cars (data) can only drive on one lane to get to one specific destination. If that lane is clogged with "Cat data," the "Dog data" can't get through.
• HHFL: We open up a multi-lane highway where cars can switch lanes or drive on multiple roads at once. The cars in the middle (the border students) act as a roundabout, letting traffic flow from the "Cat side" to the "Dog side" instantly.

The Results

The researchers tested this with real math and computer simulations (using images of handwritten numbers and complex pictures).

• When everyone has the same data: The new system is about the same as the old one.
• When data is different (Non-IID): This is where the magic happens. If the "Cat Wing" and "Dog Wing" have totally different data, the new system is 2x faster at reaching the correct answer.

Summary

The Paper says:
In our future 5G and 6G networks, devices can connect to multiple towers at once. Instead of ignoring this feature, we should use it to let devices act as bridges between different servers. This helps the AI learn faster, especially when the data is messy and unevenly distributed.

In one sentence:
We built a smarter way for AI to learn by letting devices in "no-man's-land" talk to multiple teachers at once, acting as bridges to share knowledge and speed up the whole process.

Technical Summary

1. Problem Statement

Traditional Hierarchical Federated Learning (HFL) architectures operate under a strict constraint: each client is associated with only one Edge Server (ES) at a time. While modern 5G and NextG networks increasingly utilize Coordinated Multi-Point (CoMP) transmission—allowing clients to connect simultaneously to multiple base stations (BSs/ESs) for improved signal quality and resource utilization—traditional HFL fails to leverage this capability.

In standard HFL, ESs operate in isolation between global aggregations. When data distributions across ESs are non-IID (non-independent and identically distributed), this isolation causes:

• Model Divergence: ES models accumulate biases from local gradients and drift toward localized optima.
• Slow Convergence: The lack of inter-ES knowledge sharing hinders global convergence, particularly in non-IID scenarios.
• Under-utilized Infrastructure: The potential benefits of multi-point connectivity in 5G/NextG networks remain unexploited for model training efficiency.

2. Methodology: Hybrid Hierarchical Federated Learning (HHFL)

The authors propose HHFL, a novel architecture that integrates CoMP technology into the HFL framework to enable simultaneous multi-ES communication.

Core Architecture

• Multi-Connectivity: Clients in overlapping coverage areas connect to multiple ESs simultaneously via CoMP. Clients in single-coverage areas connect to a single ES.
• Two-Level Aggregation:
  1. Client Aggregation: Upon receiving the global model, clients in overlapping areas aggregate models from all connected ESs (via simple arithmetic averaging) to initialize their local models. This acts as a "knowledge bridge" between ESs.
  2. Edge Aggregation: Clients perform local training and upload updates to all connected ESs. Each ES aggregates updates from its connected clients.
  3. Cloud Aggregation: ESs periodically upload their aggregated models to the Cloud Server (CS) for global model updates.
• Communication Design:
  • Downlink: CoMP-coordinated scheduling allocates orthogonal resource blocks (PRBs) to prevent interference, allowing clients to receive multiple models simultaneously.
  • Uplink: Clients transmit updates which are received in parallel by multiple ESs.
  • Synchronization: The design avoids heavy physical-layer synchronization (like coherent joint decoding) by using independent reception at ESs and leveraging 5G Configured Grants (CG) for pre-allocated periodic resources, minimizing latency and overhead.

Algorithm

The HHFL algorithm extends the Hier-FedAvg approach. Key mathematical modifications include:

• Client Initialization: $w_i = \frac{1}{|S_i|} \sum_{n \in S_i} w^{(n)}$, where $S_i$ is the set of connected ESs for client $i$.
• Edge Aggregation: Weighted aggregation accounts for the number of ESs a client connects to, ensuring fairness and preventing clients in overlapping regions from being overweighted in the global model.

3. Key Contributions

1. Novel Architecture: Proposal of HHFL, the first architecture to fully exploit CoMP multi-connectivity in HFL, enabling clients in overlapping regions to act as bridges for cross-ES knowledge sharing.
2. Theoretical Convergence Analysis: A rigorous mathematical proof establishing an upper bound for the convergence of HHFL. The analysis demonstrates that HHFL achieves convergence comparable to or better than HFL, with a tighter bound specifically under non-IID conditions.
3. Efficiency Validation: Extensive experiments showing that HHFL significantly outperforms traditional HFL in terms of convergence speed and resource efficiency when data is non-IID across ESs.

4. Experimental Results

The authors evaluated HHFL against traditional HFL using MNIST and CIFAR-10 datasets under various data distribution scenarios (IID and non-IID) and model types (Logistic Regression and CNN).

• Non-IID Performance:
  • In scenarios where ESs have non-IID data (e.g., each ES dominated by only 2 of 10 classes), HHFL achieved up to 2× faster convergence (in terms of training steps and wall-clock time) compared to HFL.
  • As the degree of non-IIDness increased, the performance gap widened, with HHFL maintaining robust convergence while HFL degraded significantly.
• Overlap Proportion:
  • The efficiency gain of HHFL increases with the proportion of clients in overlapping regions, eventually saturating. Even a small number of overlapping clients provides significant bridging benefits.
• Resource Efficiency:
  • Time: HHFL reduces total training time by reducing the number of required communication rounds.
  • Data Transmission: While HHFL transmits more data per round (due to multiple links), the drastic reduction in total training steps results in lower total data transmission in non-IID scenarios. In IID scenarios, HHFL incurs slightly higher overhead but offers negligible time benefits.
• Parameter Sensitivity: The advantage of HHFL becomes more pronounced as the local update steps ($E$) and edge aggregation rounds ($G$) increase, as these settings exacerbate model drift in traditional HFL.

5. Significance

• Alignment with 5G/NextG: HHFL bridges the gap between theoretical FL architectures and the physical reality of modern wireless networks (CoMP, multi-connectivity). It transforms a network feature (overlapping coverage) from a potential interference issue into a computational asset.
• Scalability: The architecture is scalable and does not require complex backhaul cooperation (e.g., coherent joint decoding), making it feasible for practical deployment in current and future cellular networks.
• Robustness: It provides a scalable solution for training FL models in heterogeneous environments where data is naturally distributed non-IID across different network nodes, a common challenge in real-world IoT and V2X applications.

In conclusion, the paper demonstrates that by rethinking the client-server association in HFL to embrace multi-connectivity, it is possible to significantly accelerate federated learning convergence and improve resource efficiency in next-generation wireless networks.