Measurement-Driven O-RAN Diagnostics with Tail Latency and Scheduler Indicators

This paper presents a measurement-driven cross-layer diagnostic methodology for O-RAN that utilizes tail latency and radio-layer indicators to identify performance degradation and device-specific behaviors under varying distances and obstruction scenarios, offering practical "degradation flags" for network troubleshooting.

The Gist

Imagine you are trying to have a video call with a friend. Sometimes the call is crystal clear, but other times, the screen freezes for a few seconds, or your voice stutters. You know something is wrong, but you don't know why. Is it your phone? Is it the Wi-Fi router? Is it the internet cable?

This paper is like a team of detectives trying to solve that mystery, but instead of a home video call, they are investigating a next-generation cellular network called O-RAN.

Here is the story of their investigation, broken down into simple concepts:

1. The Problem: The "Average" Lie

Most people check their internet speed by looking at an "average." But averages are tricky. Imagine a river: if the water is usually 1 foot deep, but there's one spot that is 100 feet deep, the "average" depth might look safe. But if you try to walk across, you'll drown in that one deep spot.

In networking, the "average" speed looks fine, but users get frustrated by rare, sudden "stalls" (long delays). The authors of this paper say: "Don't look at the average; look at the worst-case scenarios." They call this looking at the "Tail" of the data. They want to know: How often does the network choke, and how bad is it when it does?

2. The Experiment: Two Runners in a Race

To test this, the researchers set up a race track (a real cellular network) and ran two different "runners" (devices) at different distances:

• Runner A: A standard commercial Smartphone (like the one in your pocket).
• Runner B: A Modem-based device (a specialized piece of hardware often used for testing).

They tested them at three distances: 2 meters (close), 6 meters (middle), and 11 meters (far). They also created a "storm" by having people walk between the tower and the devices to block the signal, simulating a crowded room or a busy street.

3. The Big Discovery: The "Tail" Tells the Truth

When they looked at the results, they found something surprising:

• The Smartphone was like a marathon runner who is very consistent. Even when things got tough, it rarely stumbled. Its "tail" (the worst delays) was short and manageable.
• The Modem Device was like a sprinter who is fast but prone to tripping. Even though its average speed looked okay, it had a very long "tail." This means it would occasionally freeze for several seconds (a "stall"), which is terrible for a user trying to play a game or make a call.

The Lesson: If you only looked at the average speed, you would think both devices were fine. But by looking at the "tail" (the worst moments), they saw that the modem was actually much less reliable.

4. The Detective Work: Connecting the Dots

Here is where it gets really clever. Sometimes, the network looks fine from the user's perspective (the video call doesn't freeze), but the radio tower is actually struggling.

Think of it like a car engine.

• The User (App): You are driving. The car feels smooth.
• The Radio Tower (Scheduler): Under the hood, the engine is sputtering, the fuel mix is wrong, and the pistons are misfiring.
• The Fix: The car's computer (the scheduler) is working overtime to fix those misfires instantly, so you don't feel a bump.

The researchers found that even when the "video call" (latency) looked stable, the "engine" (the radio tower) was showing signs of stress (like high error rates). If they only looked at the user's experience, they would miss the fact that the engine was about to break down.

5. The Solution: The "Check Engine" Light

Because they can't always look under the hood (they don't have access to the private secrets of every device), the authors propose a new way to monitor the network.

They created a simple "Degradation Flag."
Imagine a dashboard light that only turns on when two things happen at the same time:

1. The user starts experiencing a delay (the "tail" gets long).
2. The radio tower is showing signs of trouble (like high error rates).

If both happen, the flag turns red. This tells the network engineers: "Hey, something is wrong with the connection right now, and it's not just a random glitch."

Summary

This paper teaches us three main things:

1. Averages are liars: To understand how good a network is, you have to look at the worst moments, not the average.
2. Devices matter: Different phones and modems react to the same network in very different ways.
3. Look under the hood: Sometimes the user doesn't feel a problem, but the network is struggling. By combining what the user feels with what the tower sees, we can catch problems before they become disasters.

The authors have built a simple, practical toolkit to help network operators spot these hidden problems and keep our video calls and games running smoothly.

Technical Summary

Here is a detailed technical summary of the paper "Measurement-Driven O-RAN Diagnostics with Tail Latency and Scheduler Indicators."

1. Problem Statement

The transition to O-RAN (Open Radio Access Network) architectures introduces disaggregated, software-driven components that increase flexibility but complicate performance assurance.

• The Challenge: Diagnosing performance degradations in operational O-RAN is difficult because user-perceived quality (e.g., interactive latency) results from a complex stack of interacting layers (application, transport, radio).
• The Gap: Existing empirical studies often rely on average metrics (throughput, mean latency) or single-layer Key Performance Indicators (KPIs). This approach fails to capture sporadic, high-impact "stall" events that dominate user experience. Furthermore, there is a lack of integration between application-level tail metrics (rare latency spikes) and radio-layer evidence (scheduler behavior, link adaptation).
• The Goal: To develop a cross-layer diagnostic methodology that links end-to-end latency tails with radio-layer dynamics (scheduler/link adaptation) to support root-cause triage without requiring access to proprietary internal components.

2. Methodology

The authors conducted a measurement-driven campaign using a real O-RAN testbed (OAI-based) to collect and analyze heterogeneous data.

• Experimental Setup:
  • Distances: 2m, 6m, and 11m.
  • UE Configurations: A commercial smartphone (Samsung A25) and a modem-based device (SIMCOM ASTREO).
  • Scenarios: Static conditions, spatial averaging (to mitigate small-scale fading), and a controlled dynamic obstruction scenario (15 min Line-of-Sight followed by 15 min with human movement between gNB and UE).
  • Traffic: ICMP ping packets of 30 Bytes and 1000 Bytes, sampled every 0.2 seconds.
• Data Sources:
  1. Application Layer: End-to-end Round-Trip Time (RTT) logs from ICMP pings.
  2. Radio Layer: gNB "fullstats" CSV logs containing MAC/PHY indicators (Block Error Rate - BLER, Modulation and Coding Scheme - MCS, Signal-to-Noise Ratio - SNR, retransmission counters).
• Analysis Approach:
  • Tail-Focused Characterization: Instead of averages, the study focuses on the 95th percentile (p95), median, and exceedance probabilities (e.g., probability of latency > 100ms or > 1s).
  • Windowed Correlation: Data was aggregated into short, overlapping time windows (10s windows, 5s stride) to align application latency tails with radio-layer events.
  • Cross-Layer Coupling: Statistical analysis (Spearman correlation, Kolmogorov-Smirnov tests) was used to link latency spikes with scheduler excursions (e.g., BLER spikes, MCS drops).

3. Key Contributions

1. Tail-Aware Latency Reporting: The paper demonstrates that differences between UE types (smartphone vs. modem) are primarily visible in the latency tail rather than central tendency (median). Modem-based devices exhibited significantly higher variability and rare "stall" events invisible to average-based metrics.
2. Cross-Layer Visibility: It provides evidence that radio-layer dynamics (e.g., BLER excursions, MCS adjustments) can be detected via gNB statistics even when end-to-end latency appears stable due to buffering or fast lower-layer adaptation.
3. Operational "Degradation Flags": The authors propose a lightweight, window-based diagnostic mechanism. A "degradation flag" is triggered when both (i) the latency tail exceeds a threshold and (ii) a scheduler-side excursion (e.g., elevated BLER) is present. This supports practical troubleshooting workflows.

4. Key Results

• UE-Dependent Tail Behavior:
  • Smartphones: Showed steep Cumulative Distribution Functions (CDFs) with low dispersion. The vast majority of packets had low latency.
  • Modem-based UEs: Exhibited long tails with multi-second "stall" events. Even with similar medians, the modem had significantly higher p95 values (e.g., 249.6ms vs 14.2ms for 30B packets at 6m).
• Distance and Payload Scaling:
  • Tail latency (p95) systematically increased with distance and packet size.
  • Larger packets (1000B) amplified the sensitivity of the tail, particularly for the modem-based UE.
• Dynamic Obstruction Insights:
  • In the "people moving" scenario, the smartphone's end-to-end ping latency remained stable (no clear step change) despite the introduction of blockage.
  • However, gNB-side indicators (BLER) showed localized excursions and MCS adjustments. This proves that radio stress was occurring but was masked at the application layer by link adaptation and buffering.
• Correlation:
  • A Spearman correlation of 0.53 was found between latency p95 and mean BLER in the dynamic run, confirming a link between radio reliability and application latency tails.
  • The proposed "degradation flag" successfully identified periods of radio stress that would have been missed by latency-only monitoring.

5. Significance and Impact

• Beyond Averages: The paper argues that for industrial O-RAN deployments, tail metrics are critical for understanding user experience, as rare stalls dominate perception more than average throughput.
• Interpretable Diagnostics: By correlating application logs with open gNB statistics, the study offers a method to diagnose issues without needing proprietary vendor data. This is crucial for multi-vendor O-RAN environments.
• Practical Monitoring: The proposed "degradation flags" provide a concrete, lightweight tool for network operators to detect subtle radio-layer degradations before they manifest as severe application outages.
• Future Direction: The work establishes a baseline for explainable AI and automated root-cause analysis in O-RAN, moving from descriptive statistics to actionable, cross-layer alerts.

In conclusion, this paper demonstrates that effective O-RAN diagnostics require a joint analysis of application tails and radio-layer scheduler dynamics. Relying solely on end-to-end latency or average metrics risks missing critical radio-layer instabilities that can be detected through cross-layer evidence.