Adaptive Machine Learning Framework for UAV Trajectory Optimization in O-RAN

Technical Summary: Adaptive Machine Learning Framework for UAV Trajectory Optimization in O-RAN

Problem Statement
The deployment of Unmanned Aerial Vehicles (UAVs) as Open Radio Units (O-RUs) within 6G Open Radio Access Network (O-RAN) architectures offers a promising solution for scalable and adaptive network coverage. However, optimizing UAV trajectories in dynamic, unfamiliar urban environments presents a significant challenge. Traditional Reinforcement Learning (RL) approaches, while effective for single-environment optimization, suffer from poor adaptability across diverse scenarios. When a UAV enters a new environment with different building layouts, terrain, or communication characteristics, standard RL models typically require retraining from scratch. This process is computationally expensive and time-consuming, hindering real-time operations and scalability. Furthermore, existing Transfer Learning (TL) methods often lack mechanisms to evaluate the applicability of source models to target environments, risking "negative transfer" where knowledge from dissimilar environments degrades performance rather than improving it.

Methodology
The paper proposes a novel framework that integrates enhanced Continual Transfer Learning (CTL) within the O-RAN architecture to address these limitations. The core methodology consists of three main components:

1. Model Selection Mechanism: The system maintains a library of pre-trained Deep Deterministic Policy Gradient (DDQN) models, each trained in specific urban environments (e.g., York, London, Ottawa). When a UAV encounters a new environment ($E_{new}$), the framework extracts feature vectors representing environmental characteristics (building height, density, base station distribution, map size, and UAV altitude). A composite similarity score is calculated between $E_{new}$ and all environments in the library. If the highest similarity score exceeds a predefined threshold, the most similar pre-trained model is selected for transfer learning.
2. Fallback General Model ($M_G$): In cases where no sufficiently similar model exists in the library (i.e., similarity scores fall below the threshold), the system employs a fallback model, $M_G$. This model is initially trained on a synthetic urban environment constructed by aggregating and averaging key structural parameters from all available real-world cities. This design ensures $M_G$ represents a neutral, unbiased urban setting, preventing overfitting to specific city morphologies and providing a reliable baseline for negative transfer scenarios.
3. Continual Learning (CL) Integration: The framework utilizes CL to continuously update the fallback model $M_G$. As UAVs operate in new environments and generate new policies, these models are not only stored in the library but also used to refine $M_G$ through weighted aggregation. The weights assigned to the existing $M_G$ and the new model depend on the similarity between the new environment and the original training conditions, allowing the system to progressively accumulate knowledge across diverse scenarios while mitigating catastrophic forgetting.

The entire system is embedded within the O-RAN structure. The Non-Real-Time RIC (Non-RT RIC) hosts rApps for model selection, transfer learning, and continual learning updates, while the Near-Real-Time RIC (Near-RT RIC) utilizes xApps for real-time city similarity comparison, model dispatching, and trajectory inference.

Key Contributions
The paper outlines the following specific contributions:

• Adaptive Model Selection: Development of a mechanism that selects the most appropriate pre-trained trajectory policy from a library based on an environment similarity metric, significantly reducing retraining overhead.
• Robust Fallback Model: Creation of a synthetic fallback model ($M_G$) that is continuously updated via continual learning, enabling the system to maintain baseline performance in unseen environments and progressively improve generalization.
• O-RAN Integration: Implementation of an integrated UAV trajectory system within the O-RAN architecture, utilizing xApps and rApps to enable real-time inference and continuous learning loops.
• Real-World Validation: Validation of the framework using high-fidelity ray-traced RSSI maps from real cities (York, Beijing, Ottawa, etc.), incorporating realistic propagation conditions such as diffraction and multipath effects.

Results
Simulations conducted across multiple urban environments (including York, Ottawa, and Beijing) demonstrate the efficacy of the proposed framework:

• Convergence Speed: The model selection-based transfer learning approach reduces convergence time by 44% to 56% compared to retraining from scratch (DDQN).
• Comparison with Traditional TL: The proposed method improves convergence by up to 40% compared to traditional transfer learning without model selection.
• Performance in Specific Scenarios:
  • In the York environment, where no highly similar source existed, the fallback model $M_G$ converged in 880 episodes, outperforming the best library model (940 episodes) and the DDQN baseline (1380 episodes).
  • In Ottawa, transferring from the structurally similar Manchester model achieved convergence in 750 episodes, a 44.8% improvement over the DDQN baseline.
  • In Beijing, transferring from the similar York model resulted in convergence in 590 episodes, a 45.9% improvement over the baseline.
• Negative Transfer: The study highlights that selecting models from dissimilar environments (e.g., transferring from Rosslyn to Ottawa) can lead to slower convergence and performance worse than training from scratch, validating the necessity of the similarity-based selection mechanism.
• Continual Learning Impact: While the CL-updated fallback models did not always outperform the best selected transfer models in specific instances, they demonstrated improved adaptability over time, ensuring the system remains robust as the library of environments expands.

Significance and Claims
The authors claim that this work addresses the critical gap in UAV trajectory optimization regarding adaptability in heterogeneous urban scenarios. By combining model selection with a continually updated fallback mechanism within the O-RAN framework, the system ensures that UAVs can rapidly adapt to new environments without the prohibitive costs of retraining from scratch. The paper emphasizes that this approach not only accelerates learning but also ensures robust and reliable operation by preventing negative transfer. The integration of real-world ray-tracing data and the O-RAN architecture positions the framework as a practical step toward scalable, intelligent 6G network deployment. The authors note that future work will explore further enhancements in model selection strategies and energy-efficient trajectory planning.