⚡ electrical engineering

Distributed Precoding for Cell-free Massive MIMO in O-RAN: A Multi-agent Deep Reinforcement Learning Framework

This paper proposes a distributed, multi-agent deep reinforcement learning framework for cell-free massive MIMO in O-RAN that efficiently determines precoding matrices to maximize aggregate throughput and meet user rate requirements while significantly reducing signaling overhead compared to centralized schemes and outperforming existing distributed methods.

Original authors: Mohammad Hossein Shokouhi, Vincent W. S. Wong

Published 2026-05-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mohammad Hossein Shokouhi, Vincent W. S. Wong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a massive wireless network as a giant, bustling concert hall. In this hall, there are hundreds of tiny speakers (called O-RUs) scattered everywhere, and thousands of audience members (called users) trying to hear their favorite songs clearly.

In the old days, the concert hall had a single, giant sound system in the center. But that system struggled when the hall got too big or too crowded; some people heard the music perfectly, while others in the back or corners heard nothing but static.

This paper proposes a new way to run the concert: Cell-Free Massive MIMO. Instead of one big speaker, every single tiny speaker works together to beam sound directly to specific audience members. The goal is to make sure everyone hears their song loudly and clearly, without the sound from one person's song drowning out their neighbor's (this is called interference).

However, getting hundreds of speakers to coordinate perfectly is incredibly hard. It's like trying to get a choir of 1,000 people to sing in perfect harmony without a conductor shouting instructions to everyone at once. If they all try to talk to each other to coordinate, the communication lines get clogged (too much signaling overhead). If they just guess, the music sounds messy.

The Problem: The "Conductor" vs. The "Chaos"

The researchers looked at two existing ways to solve this:

The Centralized Conductor: One super-brain (a central computer) calculates exactly what every speaker should do. This works well for small groups, but if the group gets too big, the brain gets overwhelmed, and the instructions take too long to arrive.
The Independent Singers: Each speaker decides what to do on its own. This is fast, but without coordination, they often step on each other's toes, causing a mess of noise.

The Solution: A Smart, Multi-Tiered Team

The authors propose a new framework that fits into a modern network architecture called O-RAN (Open Radio Access Network). Think of O-RAN as a smart management system that allows different parts of the network to talk to each other efficiently.

Their solution is a Multi-Agent Deep Reinforcement Learning (DRL) framework. Here is how it works, using a creative analogy:

1. The "Expert" and the "Student"

Imagine a master conductor (the Iterative Algorithm) who knows the perfect way to arrange the music but is very slow and tired. He has to calculate complex math for every single note.

The Innovation: Instead of asking the master to do the math every second, the researchers trained a team of AI students (the DRL Agents) to watch the master and learn his "intuition."
The AI doesn't try to calculate the whole song from scratch. Instead, it learns to make a few smart, quick decisions (like adjusting the volume or timing) based on what the master would have done. This is called using "expert knowledge."

2. The Three-Speed Clock (Multi-Timescale)

The paper introduces a clever way to handle time, like a clock with three different speeds:

The Slow Clock (Non-RT Loop): Once in a while (every few seconds), the "Master" (a central AI trainer) updates the students' general strategy. It teaches them how to handle big changes in the crowd.
The Medium Clock (Near-RT Loop): Every few milliseconds, the students (running on a central controller) quickly adjust the "volume knobs" and "timing" for the whole group based on the current crowd density.
The Fast Clock (RT Loop): Every millisecond, the local speakers (O-RUs) make tiny, instant adjustments to their beams to hit the moving audience members perfectly. They use the "volume knobs" set by the students to do this instantly.

3. The "Secret Handshake" (Distributed Coordination)

Instead of every speaker shouting to every other speaker (which clogs the network), the speakers only talk to their immediate neighbors and a local manager.

They share just enough information to avoid stepping on each other's toes.
This is like a group of dancers who know the choreography. They don't need to ask the whole room what to do; they just watch the person next to them and the music, and they know exactly where to move.

The Results: What Did They Find?

The researchers ran simulations to see how well this new "AI-conducted choir" performed compared to the old methods:

Better Sound Quality: The new system delivered up to 50% more total data (throughput) than the old distributed methods where speakers acted alone.
As Good as the Master: It performed almost as well as the "Centralized Conductor" (the perfect but slow method) but was much faster.
Less Traffic: Because the speakers didn't need to shout instructions to the central brain constantly, the network traffic (signaling overhead) was reduced by up to 99.8%. It's like replacing a thousand phone calls with a single text message.
Adaptable: If a user suddenly needs a faster connection (like switching from listening to a podcast to watching a 4K movie), the system automatically adjusts to meet that need without crashing.
Robust: Even if the speakers can't hear the audience perfectly (due to noise or "imperfect channel estimation"), the AI system is surprisingly good at guessing the right moves, often outperforming the traditional mathematical methods in noisy environments.

In Summary

This paper presents a smart, scalable way to manage a massive wireless network. Instead of relying on a slow, overloaded central brain or a chaotic group of independent speakers, they created a team of AI agents that learn from expert math but act quickly and locally. They coordinate just enough to avoid noise, ensuring everyone in the "concert hall" gets a clear, high-quality signal, even when the crowd is huge and moving fast.

1. Problem Statement

The paper addresses the challenge of precoding in Cell-free Massive MIMO systems integrated within the Open Radio Access Network (O-RAN) architecture.

Context: In cell-free massive MIMO, multiple Open Radio Units (O-RUs) collaboratively serve users to ensure uniform coverage and high capacity. Precoding is critical for steering beams and mitigating inter-user interference.
The Challenge:
- Centralized schemes (e.g., Centralized Regularized Zero-Forcing, C-RZF) offer high performance but suffer from scalability issues, high computational complexity, and excessive signaling overhead on the E2 interface (between O-DUs and RICs).
- Fully distributed schemes (e.g., Distributed RZF) reduce overhead but often lead to suboptimal performance due to high inter-O-RU interference and lack of global coordination.
- Existing ML approaches: Direct Deep Reinforcement Learning (DRL) to determine precoding matrices often fails in dense deployments due to high-dimensional state/action spaces, leading to convergence issues. Furthermore, many existing methods focus solely on maximizing aggregate throughput without guaranteeing minimum data rate requirements for individual users.
Goal: Develop a distributed, scalable, and low-latency precoding framework that guarantees user minimum rate requirements, maximizes aggregate throughput, and minimizes signaling overhead within the O-RAN hierarchy.

2. Methodology

The authors propose a hybrid multi-timescale framework that combines expert knowledge from iterative optimization algorithms with Multi-Agent Deep Reinforcement Learning (DRL). The solution leverages the hierarchical O-RAN architecture (Non-RT RIC, Near-RT RIC, and O-DUs).

A. Mathematical Formulation

Optimization Problem: The authors formulate a non-convex optimization problem to maximize aggregate throughput subject to minimum rate constraints ( $R_{min}$ ) and transmit power constraints.
Transformation: The problem is reformulated as an equivalent Weighted Minimum Mean Square Error (WMMSE) problem.
Iterative Solution: A Block Coordinate Descent (BCD) approach is used to iteratively update:
1. Receive filters ( $U$ )
2. Weight matrices ( $W$ )
3. Precoding matrices ( $V$ )
Key Insight: In cell-free systems, precoding matrices are coupled across O-RUs. However, the receive filters ( $U$ ) and weights ( $W$ ) depend on effective channel sums which exhibit "channel hardening," making them less sensitive to rapid fluctuations than the instantaneous precoding matrices.

B. The Proposed Multi-Agent DRL Framework

Instead of using DRL to directly predict high-dimensional precoding matrices (which is computationally expensive and hard to train), the framework uses DRL to learn the low-dimensional intermediate variables ( $U$ and $W$ ) derived from the iterative algorithm.

Multi-Agent Architecture:
- Agents: One DRL agent is assigned to each user.
- Observation: Each agent observes the effective interference channels ( $\Xi_{k,i}$ ) from a subset of interfering users ( $I_k$ ) that share similar large-scale fading profiles.
- Action: The agent outputs the receive filter ( $U_k$ ) and the Cholesky factor of the weight matrix ( $L_k$ , where $W_k = L_k L_k^H$ ). This reduces the action space dimension significantly.
- Algorithm: A multi-agent extension of the Soft Actor-Critic (SAC) algorithm is used to maximize a combination of reward (throughput) and policy entropy.

C. Multi-Timescale O-RAN Deployment

The framework is distributed across different O-RAN components based on their operational timescales:

Non-RT Loop (>1 sec):
- Component: Non-RT RIC (rApps).
- Task: Training the DRL models (Actor and Critic networks) and updating parameters.
Near-RT Loop (10 ms – 1 sec):
- Component: Near-RT RIC (xApps).
- Task: Agents determine $U_k$ and $W_k$ for each user based on current channel statistics. These are sent to O-DUs.
Real-Time (RT) Loop (<1 ms):
- Component: O-DUs (dApps).
- Task: Using the received $U_k$ and $W_k$ , along with instantaneous Channel State Information (CSI) and a learned Lagrange multiplier ( $\xi_l$ ) for power constraints, the O-DUs compute the final precoding matrices ( $V_{k,l}$ ) using a closed-form update equation.
- Inter-O-DU Communication: A new D2 interface is proposed to allow O-DUs to exchange necessary channel/precoding information for coupled O-RUs, minimizing the load on the E2 interface.

3. Key Contributions

Novel Optimization Formulation: Formulated a non-convex precoding problem for cell-free massive MIMO with coherent joint transmission, reformulated as a WMMSE problem, and solved via BCD.
Expert-Guided DRL: Proposed a hybrid framework where DRL learns low-dimensional intermediate variables ( $U, W$ ) rather than high-dimensional precoding matrices. This leverages optimization theory to guide the learning process, improving convergence and stability.
Hierarchical O-RAN Implementation: Designed a practical deployment across Non-RT, Near-RT, and RT loops, utilizing dApps for real-time precoding and xApps/rApps for optimization and training.
Dynamic Adaptability: The framework dynamically adapts to changes in user minimum data rate requirements ( $R_{min}$ ) by updating Lagrange multipliers in real-time without retraining.
Signaling Reduction: Introduced a distributed mechanism that drastically reduces signaling overhead on the E2 interface compared to centralized schemes.

4. Performance Results

Extensive simulations were conducted comparing the proposed framework against C-RZF, D-RZF, WMMSE, and various DRL baselines (E2E-DRL, DRL-WMMSE).

Aggregate Throughput:
- Outperforms D-RZF by up to 24.4%.
- Outperforms WMMSE by up to 35.8%.
- Outperforms End-to-End DRL by up to 50.2%.
- Achieves performance comparable to Centralized RZF (C-RZF) but with significantly lower complexity.
Signaling Overhead:
- Reduces E2 interface signaling overhead by up to 99.81% compared to C-RZF. This is because only low-dimensional intermediate variables are exchanged, not full channel matrices.
Scalability:
- Computational complexity scales efficiently with the number of users and O-RUs, unlike centralized methods which scale cubically.
- Inference latency per Near-RT loop is 6.38 ms, well within the 10 ms target.
Robustness:
- Mobility: The framework maintains performance even at high user velocities (up to 45 km/h), with minimal degradation (<4%).
- Imperfect CSI: Under severe channel estimation errors, the proposed learning-based method outperforms the centralized analytical method (C-RZF).
- Fairness: By adjusting $R_{min}$ , the framework can balance between maximizing aggregate throughput and ensuring fairness for cell-edge users.

5. Significance

This paper bridges the gap between theoretical optimization and practical deployment in next-generation wireless networks.

Practicality: It demonstrates how O-RAN's flexible architecture can host complex AI/ML solutions (DRL) while adhering to strict real-time constraints.
Efficiency: It solves the "curse of dimensionality" in DRL for wireless systems by using domain knowledge (iterative algorithms) to reduce the learning space.
Future-Proofing: The framework is designed to handle dynamic network conditions (user mobility, changing QoS requirements) and imperfect channel information, making it a viable candidate for 6G cell-free networks.

In summary, the paper presents a robust, scalable, and efficient solution for cell-free massive MIMO precoding that leverages the synergy between optimization theory and multi-agent reinforcement learning within the O-RAN ecosystem.