Leveraging Multi-Agent System (MAS) and Fine-Tuned Small Language Models (SLMs) for Automated Telecom Network Troubleshooting

Technical Summary: Leveraging Multi-Agent System (MAS) and Fine-Tuned Small Language Models (SLMs) for Automated Telecom Network Troubleshooting

Problem Statement

Telecommunication networks are evolving into highly dynamic, heterogeneous environments characterized by increasing scale and complexity. This evolution makes management, operation, and optimization significantly challenging. While AI/ML has been applied to fault detection, the diagnostic and remediation processes remain heavily reliant on Subject Matter Experts (SMEs) to manually correlate diverse data sources (performance metrics, alarms, logs, configurations) and determine root causes. Existing automated approaches face critical limitations:

• Scalability and Generalization: Traditional rule-based or data-driven methods struggle to scale across heterogeneous deployments and often require extensive labeled data.
• LLM Deployment Barriers: While Large Language Models (LLMs) offer strong reasoning capabilities, their practical deployment in telecom is hindered by high operational costs, data privacy risks regarding sensitive network telemetry, and substantial capital expenditures (CapEx) for hosting large models.
• Lack of End-to-End Automation: Current solutions often lack integrated workflows that seamlessly connect detection, diagnosis, planning, and execution.

Methodology

The authors propose a Multi-Agent System (MAS) that orchestrates a workflow combining the general reasoning of an LLM with the efficiency of a fine-tuned Small Language Model (SLM). The system operates within a ReAct (Reasoning, Execution, Action) loop and is built on the Hypha framework, which facilitates agent-to-agent interactions via Remote Procedure Calls (RPC).

1. Multi-Agent Architecture
The MAS comprises six specialized agents:

• Orchestrator: A conversational agent (powered by an LLM, e.g., GPT-4o mini in experiments) that acts as the central controller, interpreting faults and delegating tasks.
• Solution Planner: A critical agent responsible for generating stepwise troubleshooting procedures. To ensure domain accuracy, this agent is powered by a fine-tuned SLM and integrated with a knowledge graph built from internal operational documents (PM, FM, CM data) using HippoRAG (a retrieval backbone superior to standard RAG and GraphRAG in internal evaluations).
• Root Cause Analysis (RCA) Agent: Correlates faults across multiple data sources (PM, logs, alarms) to identify underlying causes, utilizing causal intervention sequences.
• Executor: Refines the solution planner's output by adding specific parameters (node IDs, time intervals) and implements actions after validation.
• Data Retriever: Accesses diverse databases (PostgreSQL for RAN, VictoriaMetrics for Core, OpenSearch for logs/alarms).
• Dashboard Display: Visualizes results for SME review.

The system incorporates a Human-in-the-Loop (HITL) mechanism: approximately 80% of the workflow is autonomous, 15% involves approval gates for critical actions to prevent impact on live traffic, and 5% requires human judgment for business prioritization.

2. Fine-Tuning the Solution Planner (SLM)
To address the cost and privacy limitations of external LLMs, the authors fine-tune an SLM (specifically Unsloth/DeepSeek-R1-Qwen-3-8B) to serve as the domain-grounded solution planner. The training pipeline involves:

• Supervised Fine-Tuning (SFT): A warm-up phase using labeled QA pairs to learn domain context and formatting.
• Reinforcement Fine-Tuning (RFT): Utilizing the Transformers Reinforcement Learning (TRL) framework with Group Relative Policy Optimization (GRPO) and Low-rank adaptation (LoRA).
• Reward Design: The RFT process employs two reward categories:
  • Format Rewards: Regex-based checks for XML tag usage and step demarcation.
  • RAGAS-based Rewards: Customized for telecom to evaluate Completeness (coverage of relevant counters/alarms), Relevancy (alignment with the query), and Groundedness (verification against retrieved context to reduce hallucinations).
• Optimization: The authors implemented uniform document chunking to manage context length and memory usage, significantly improving training throughput and convergence.

Key Contributions

1. Hybrid MAS Architecture: An end-to-end framework that integrates LLM-based orchestration with specialized agents for retrieval, analysis, and execution, reducing inter-agent communication overhead.
2. Domain-Grounded SLM Planner: A novel approach to fine-tuning an SLM via SFT and RFT to generate executable, domain-specific remediation plans, addressing the cost and privacy constraints of external LLMs.
3. Advanced Retrieval Integration: The use of HippoRAG to enhance grounding by integrating LLM-extracted triples with knowledge graphs, outperforming standard RAG and GraphRAG in internal evaluations.
4. Comprehensive Evaluation: A rigorous assessment across both Radio Access Network (RAN) and Core network domains, including ablation studies on retrieval backends and fine-tuning stages.

Experimental Results

The framework was evaluated on RAN (power systems, heartbeat failures, capacity congestion) and Core (PDU session degradation) scenarios.

• Operational Efficiency: The autonomous MAS achieved a six-fold reduction in mean troubleshooting time per node compared to human engineers and improved troubleshooting accuracy by 10%.
• Model Performance:
  • The fine-tuned 8B SLM achieved a Total Reward of 10.51 (compared to 5.96 for the base model) and a standard deviation of 0.38 (compared to 2.17), indicating significantly higher stability and consistency.
  • The fine-tuned model generated solutions comparable to GPT-4o mini, providing detailed counter-level steps, whereas the base model produced generic, non-actionable responses.
• Ablation Findings:
  • Retrieval: HippoRAG yielded the highest RAGAS rewards (5.19) and lowest variance compared to Standard RAG (3.61) and GraphRAG (4.08).
  • Training Stages: While SFT alone improved rewards (8.31), the addition of RFT was necessary to achieve counter-level detail and further reduce variance (0.38).

Significance

The paper claims that this architecture provides a scalable, efficient, and privacy-preserving solution for telecom network automation. By combining the orchestration power of LLMs with the cost-effectiveness and domain adaptability of fine-tuned SLMs, the framework significantly reduces reliance on human SMEs and external LLM providers. It demonstrates that autonomous network operations can turn network complexity into a competitive advantage by delivering faster troubleshooting, lower operational costs, and improved customer experience, while maintaining strict adherence to data privacy and operational safety through HITL controls.