⚡ electrical engineering

Scalable machine learning-based approaches for energy saving in densely deployed Open RAN

This paper proposes scalable Deep Reinforcement Learning (DRL) solutions, including a federated approach, for Open RAN networks that achieve significant energy savings and faster convergence while maintaining quality of service in densely deployed base station scenarios.

Original authors: Xuanyu Liang, Ahmed Al-Tahmeesschi, Swarna Chetty, Cicek Cavdar, Berk Canberk, Hamed Ahmadi

Published 2026-04-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Xuanyu Liang, Ahmed Al-Tahmeesschi, Swarna Chetty, Cicek Cavdar, Berk Canberk, Hamed Ahmadi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a bustling city at night. The streetlights are the Base Stations (or Radio Units) that keep our phones connected. During the day, when everyone is rushing around, you need every single light on to make sure no one trips in the dark. But at 3:00 AM, when the streets are empty, keeping all those lights blazing is a huge waste of electricity.

This is exactly the problem mobile networks face. As we add more and more towers to handle our data needs, they consume massive amounts of energy. The goal of this paper is to figure out how to turn off the "lights" (sleep mode) when they aren't needed, without leaving anyone in the dark.

Here is the story of how the researchers solved this, explained simply:

1. The Old Way vs. The New Way

The Old Way (Traditional Networks):
Think of a traditional network like a rigid, old-school building. You can't easily turn off just the lights in the hallway without shutting down the whole floor. It's too complicated to control individual parts.

The New Way (Open RAN):
The researchers are using a new architecture called Open RAN. Imagine this as a "smart building" with a central brain (the Controller) and many smart sensors (the Radio Units). This brain can talk to the sensors and say, "Hey, nobody is in Room 4, turn off the lights there, but keep them on in Room 1."

2. The Challenge: Too Many Choices

The tricky part is that the "brain" has to make these decisions instantly, thousands of times a day.

If you have 6 lights, there are 64 possible combinations of on/off.
If you have 24 lights, there are 16 million combinations!

Trying to calculate the perfect answer for 16 million options every second is like trying to solve a Sudoku puzzle while running a marathon. It's too slow and too heavy for a single computer to handle.

3. The Solution: Teaching a Robot to Think (Machine Learning)

Instead of doing the math manually, the researchers taught a Robot (AI) to learn by doing. This is called Deep Reinforcement Learning (DRL).

The Robot's Job: It looks at the city (the network), sees where people are walking, and decides which lights to turn on or off.
The Reward: If it saves electricity and keeps everyone connected, it gets a "gold star" (positive reward). If it turns off a light while someone is trying to make a call, it gets a "frown" (penalty).

They tried three different ways to train this robot:

The "List Maker" (DQNMA): The robot tries to memorize every single possible combination of lights. Problem: When the city gets big, the list is too long to memorize.
The "One-by-One" (DQNSA): The robot changes just one light at a time. Problem: It's too slow to react to a sudden crowd.
The "Smooth Operator" (TD3): This is the winner. Instead of thinking in rigid "On/Off" switches, this robot thinks in "dimmer switches" (continuous values) and then snaps them to On/Off. It's much faster and smarter at handling big cities.

4. The Big Breakthrough: The "Team of Teachers" (Federated Learning)

Even the "Smooth Operator" robot struggles when the city gets huge (like a whole country). It's like trying to teach one student to know everything about every street in the world. It takes too long, and the computer gets tired (high energy cost).

So, the researchers came up with a Federated Learning approach. Imagine a school with many classrooms:

The Local Teachers (xApps): Each classroom (a specific neighborhood) has its own teacher. They teach their students based only on what happens in their own neighborhood. They don't need to know about the traffic in a different city.
The Principal (Aggregator/rApp): Once a week, the teachers meet with the Principal. They don't share their students' notebooks (private data); they just share their lesson plans (what they learned).
The Global Wisdom: The Principal combines the best parts of all the lesson plans to create a "Super Lesson Plan" and sends it back to all the teachers.

Why is this amazing?

Speed: The teachers learn faster because they aren't waiting for data from the whole world.
Privacy: No one sees the raw data of other neighborhoods.
Energy: It saves a massive amount of electricity because the computers aren't working as hard.

5. The Results: A Brighter, Greener Future

The researchers tested this in a virtual city. Here is what happened:

Faster Learning: The "Team of Teachers" (Federated) learned 43% faster than the single "Super Teacher" (Centralized).
Huge Savings: The network saved over 50% of its energy by smartly turning off unused lights.
Less Waste: The training process itself used 37% less electricity compared to the old way.
No Blackouts: Crucially, even with the lights turned off, the "quality of service" (the ability to make calls and stream video) stayed perfect.

The Bottom Line

This paper shows that by using smart AI and a "teamwork" approach (Federated Learning), we can make our mobile networks much greener. We can turn off the "lights" when they aren't needed, saving money and helping the planet, all while making sure your phone stays connected. It's like having a smart city that knows exactly when to dim the lights, without anyone having to manually flip the switches.

1. Problem Statement

The rapid densification of Base Stations (BSs) and Radio Units (RUs) to meet 5G/6G data rate requirements has led to a significant surge in network energy consumption, with BSs accounting for 73–77% of total network energy usage. While keeping RUs active during off-peak hours ensures coverage, it results in massive energy waste.

Core Challenge: The problem of determining which RUs to switch to "sleep mode" while maintaining Quality of Service (QoS) is an NP-hard combinatorial optimization problem due to the exponential number of on/off configurations and temporal coupling.
Scalability Issue: Traditional centralized Machine Learning (ML) approaches struggle in large-scale, geographically distributed Open RAN (O-RAN) networks due to high communication overhead, data heterogeneity, and the "curse of dimensionality" in action spaces as the number of RUs increases.
Objective: To develop a scalable, energy-efficient control mechanism that dynamically switches RUs between active and sleep modes, minimizing total network energy consumption while satisfying user data rate constraints.

2. Methodology

The authors propose a framework leveraging the O-RAN architecture, specifically utilizing the Near Real-Time RIC (Near-RT RIC) for fast control (xApps) and the Non-Real-Time RIC (Non-RT RIC) for long-term policy aggregation (rApps).

A. System Model & Formulation

MDP Formulation: The problem is modeled as a Markov Decision Process (MDP).
- State Space ( $s_t$ ): Includes user data rates, previous RU activation states, RU load (PRB utilization), and UE spatial coordinates (to capture mobility).
- Action Space ( $a_t$ ): Binary vectors indicating the active/sleep status of each RU.
- Reward Function ( $r_t$ ): A weighted sum penalizing total energy consumption and the number of users failing to meet minimum data rate thresholds (QoS violation).
Power Model: Total power includes fixed power (active/sleep), load-dependent transmission power (proportional to PRB usage), and transition power (cost of waking up an RU).

B. Centralized DRL Approaches

Three centralized algorithms were developed and compared to handle the action space:

DQN-Multiple Action (DQNMA): Treats the entire RU configuration as a single discrete action. The action space size is $2^M$ (exponential), making it intractable for large $M$ .
DQN-Single Action (DQNSA): Controls only one RU per time step. Reduces action space to linear size ( $2M$ ) but limits adaptability to rapid network changes.
TD3 (Twin Delayed DDPG): A continuous control algorithm. The actor outputs continuous values in $[0, 1]$ , which are discretized via a threshold (e.g., $>0.5$ ) to determine sleep/active states. This avoids exponential action growth and allows for fine-grained policy learning.

C. Federated DRL (Fed-DRL) Architecture

To address scalability, the authors propose a Federated Reinforcement Learning (FRL) framework aligned with O-RAN hierarchy:

Local Agents (xApps): Located in Near-RT RICs for specific geographic regions. They train local TD3 (or DQN) models using local traffic and mobility data without sharing raw data.
Global Aggregator (rApp): Located in the Non-RT RIC. It periodically aggregates model parameters (Actor and Critic weights) from local agents using a weighted averaging strategy (FedAvg) to update the global policy.
Workflow: Local training $\rightarrow$ Parameter upload $\rightarrow$ Global aggregation $\rightarrow$ Policy redistribution. This preserves privacy and reduces backhaul load.

3. Key Contributions

O-RAN Compliant Framework: Designed an energy control system that strictly adheres to O-RAN functional splits (xApps for real-time control, rApps for orchestration) and standardized interfaces (A1, E2).
MDP Formulation for RU Sleep: Formulated the joint RU sleep control problem as an MDP considering traffic load, user mobility, and spatial distribution, with a reward function balancing energy and QoS.
Scalable Federated Architecture: Proposed a Fed-TD3 solution where local agents train independently and synchronize via a global aggregator. This shifts the agent granularity from per-BS to per-region, improving generalization across heterogeneous deployments.
Comprehensive Evaluation: Conducted extensive simulations comparing centralized vs. federated approaches across varying network sizes (6 to 24 RUs) and layouts (single vs. composite multi-region).

4. Simulation Results

The authors evaluated the models using 3GPP UMi channel models with user mobility in 500m and 1000m regions.

Algorithm Performance (Centralized):
- TD3 consistently outperformed DQN variants (DQNMA and DQNSA) in terms of reward stability and convergence speed.
- TD3 achieved up to 6% additional energy savings compared to DQN models in dense scenarios by better handling the continuous nature of control decisions.
Federated vs. Centralized:
- Convergence Speed: Fed-TD3 converged 43.75% faster (approx. 859 episodes) than centralized TD3 (approx. 1421 episodes).
- Energy Savings: The federated approach achieved >50% network energy savings compared to the theoretical maximum (all RUs active).
- Training Energy Efficiency: Fed-TD3 reduced the total energy consumed during the training process by 37.4% compared to centralized TD3, highlighting the computational efficiency of distributed training.
- Generalization: Federated models trained on separate sub-regions generalized well to composite multi-region scenarios, often outperforming centralized models trained on the same combined data.
Trade-offs: While Fed-TD3 prioritized QoS (keeping more RUs active to ensure user satisfaction), Fed-DQN models were more aggressive in sleeping RUs, sometimes at the cost of slight QoS degradation.

5. Significance

This paper demonstrates that Federated Deep Reinforcement Learning is a viable and superior solution for energy management in next-generation O-RAN networks.

Scalability: It overcomes the limitations of centralized ML in ultra-dense networks by distributing the learning load.
Privacy & Efficiency: It eliminates the need to share raw user data and significantly reduces the computational and communication overhead required for training.
Practical Deployment: By mapping the learning workflow directly to O-RAN's existing xApp/rApp hierarchy, the proposed solution is ready for practical implementation in 5G/6G networks, offering a path toward sustainable, green telecommunications infrastructure.