💻 computer science

Dynamic Sampling for Telemetry in Microservices: A Reinforcement Learning and Entropy-Based Approach

This paper introduces RADAR, a Reinforcement Learning and entropy-based agent that dynamically optimizes microservices telemetry sampling to drastically reduce resource consumption while preserving critical observability and data quality.

Original authors: Renan Martins Alves, Jéferson Campos Nobre, Juliano Araujo Wickboldt

Published 2026-10-19

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Renan Martins Alves, Jéferson Campos Nobre, Juliano Araujo Wickboldt

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Modern software often lives not as a single, massive program, but as a swarm of small, independent services talking to one another across a network. This approach, known as microservices, allows companies to update parts of their applications quickly and scale them up or down as needed. However, this flexibility comes with a hidden cost: it becomes incredibly difficult to see what is happening inside the system when things go wrong. To solve this, engineers use a technique called distributed tracing. Imagine a customer placing an order on a website; that single request might bounce through a dozen different services, checking inventory, calculating taxes, and processing payments. Distributed tracing follows that request from start to finish, creating a detailed map of its journey. This map is essential for finding bugs and understanding performance issues, but it generates a massive amount of data. If a system processes millions of requests, collecting every single trace would overwhelm the network, consume huge amounts of computing power, and fill up storage drives at an unsustainable rate.

The challenge, then, is to collect enough data to see the problems without drowning in the noise of routine, successful operations. Traditional methods often rely on simple rules, such as recording only one out of every hundred requests, or waiting until an error occurs to start recording. These approaches are often too rigid; they might miss rare but critical failures, or they might waste resources recording mundane traffic that offers no new insights. Researchers at the Federal University of Rio Grande do Sul in Brazil have developed a new way to handle this dilemma. They created a system called RADAR, which acts as an intelligent, self-adjusting filter for this data. Instead of using fixed rules, RADAR learns which traces are worth keeping by constantly testing different combinations of sampling rules and observing the results.

The core idea behind RADAR is to treat the collection of data as a learning process. The system uses a method called reinforcement learning, where a computer agent makes decisions, sees what happens, and adjusts its strategy to get better results over time. In this case, the agent's goal is to maximize the "information value" of the data it keeps while minimizing the amount of data it stores. To measure information value, the researchers used a concept from information theory known as entropy. In simple terms, entropy measures how diverse or unpredictable a set of data is. A collection of traces that are all identical has low entropy and low value, because they tell you nothing new. A collection that includes many different, rare, and unusual paths has high entropy and high value, because it reveals the complex behaviors and potential failures of the system. RADAR uses this measure of diversity as a reward signal. When the agent selects a set of rules that captures a wide variety of unique behaviors without collecting too much data, it receives a high reward and learns to repeat that choice.

To test this idea, the researchers built a simulated online store running on a cluster of computers, mimicking the complex environment of a real-world application. They introduced various types of traffic, including normal browsing and specific scenarios designed to trigger errors or unusual behavior. They then let the RADAR agent run for several hours, allowing it to experiment with different rules. Some rules focused on capturing slow requests, others on errors, and others on specific business events like high-value purchases. The agent would switch between these rules, observing how many traces were collected and how diverse they were. Over time, the agent learned to find the perfect balance. It discovered that it did not need to record every single request to understand the system. Instead, by focusing on the most diverse and informative paths, it could maintain a clear picture of the system's health while drastically cutting down on the volume of data.

The results of this experiment were striking. Compared to a system that recorded every single trace, RADAR reduced the amount of data sent over the network by 97.4 percent. It also reduced the computing power required to process that data by 99.0 percent. These savings are significant because they mean the system can run much cheaper and faster. Yet, the researchers were careful to ensure that this efficiency did not come at the cost of missing important information. They compared the data collected by RADAR against the full dataset to see if it had captured the rare and critical events. They found that RADAR retained approximately 85.6 percent of the rare trace patterns—those unique, one-off events that often signal the beginning of a serious problem. Furthermore, the average diversity of the data that was kept increased by about 25 percent, meaning the remaining data was much more useful for diagnosis than a random sample would have been.

This approach represents a shift from static, human-defined rules to dynamic, autonomous management. Traditional systems require engineers to manually decide which rules to apply, a task that becomes difficult as the system changes and grows. RADAR, by contrast, continuously adapts to the current state of the system. It does not need to be told what an error looks like in advance; it simply learns to recognize that a diverse set of behaviors is more valuable than a repetitive one. The system operates entirely within the standard tools used by modern software engineers, making it compatible with existing infrastructure without requiring invasive changes to the application code itself.

The study demonstrates that it is possible to drastically reduce the burden of monitoring complex systems without sacrificing the ability to see what is wrong. By using a mathematical measure of diversity to guide the collection process, the researchers showed that an intelligent agent can learn to filter out the noise and keep the signal. While the experiment was conducted in a controlled environment, the results suggest a viable path forward for managing the growing complexity of cloud-based applications. The work highlights that the future of system monitoring may not lie in collecting more data, but in collecting smarter data, guided by agents that understand the value of information in real time.

Technical Summary: Dynamic Sampling for Telemetry in Microservices (RADAR)

1. Problem Statement

The transition to microservices architectures has increased application scalability but significantly complicated system observability. In distributed environments, a single user request traverses multiple independent services, making distributed tracing essential for debugging and failure diagnosis. However, collecting full traces at scale is often infeasible due to prohibitive costs in network bandwidth, CPU processing, and storage.

Traditional sampling strategies face distinct limitations:

Head-based sampling: Decides to discard data at the start of a transaction, often missing rare errors or anomalies that manifest later in the flow.
Tail-based sampling: Allows for informed decisions based on the full trace but relies on rigid, manually configured rules that cannot adapt to the dynamic load and behavioral variations typical of cloud environments.
Existing advanced solutions: Approaches like Hindsight (reactive anomaly triggers) or clustering-based methods often suffer from rigid trigger definitions, lack of compatibility with modern standards like OpenTelemetry, or require invasive instrumentation.

The central challenge addressed is how to significantly reduce the volume of collected traces while preserving the high informational value required for effective observability, without relying on static heuristics or prior data labeling.

2. Methodology: The RADAR Framework

The authors propose RADAR (Reinforcement Learning Agent for Dynamic And Relevant trace sampling), an autonomous agent integrated into the OpenTelemetry ecosystem. RADAR dynamically adjusts sampling policies by combining Reinforcement Learning (RL) with Shannon entropy.

2.1 System Architecture

RADAR operates as a closed feedback loop instantiating the MAPE-K (Monitor, Analyze, Plan, Execute, Knowledge) autonomic model:

Monitor: An OpenTelemetry Collector receives traces from a distributed application and exports them to a Tracing Backend (Jaeger) and Database (Elasticsearch).
Analyze: A Metrics Extractor queries the database to compute statistical features, specifically the Shannon entropy of the collected traces and the total volume.
Plan: A Reinforcement Learning Agent uses these statistics to calculate a reward and select a new subset of sampling policies from a predefined catalog.
Execute: A Rules Manager translates the agent's decision into an updated configuration for the OpenTelemetry Collector, applying the new rules to the live environment via Kubernetes.

2.2 Problem Formalization

The adaptive sampling challenge is formalized not as a traditional Markov Decision Process (MDP), but as a combinatorial Multi-Armed Bandit problem.

Stateless Environment: The agent does not observe a changing system state (e.g., current CPU load) but rather selects actions based on the history of rewards.
Action Space ( $A$ ): A discrete, combinatorial space where the agent selects a binary vector representing the activation ($1$) or deactivation ($0$) of $N$ independent sampling policies (e.g., rules based on latency, error codes, or business attributes).
Algorithm: The system adapts the REINFORCE policy-gradient algorithm to learn the optimal probability distribution for selecting rule combinations.

2.3 Reward Function

The core innovation is a reward function ( $R$ ) that balances informational richness against operational cost:
$R = \alpha \cdot H(X) - \beta \cdot P(n)$

$H(X)$ (Entropy): Shannon entropy calculated over the frequency distribution of unique trace representations. Higher entropy indicates a diverse set of execution paths (including rare/anomalous ones) rather than repetitive, trivial paths.
$P(n)$ (Volume Penalty): A sigmoid-shaped function that penalizes the number of collected traces ( $n$ ) as it approaches a threshold ( $C$ ). This ensures the system stays within resource budgets.
$\alpha$ and $\beta$ : Hyperparameters controlling the trade-off between information diversity and resource savings.

2.4 Entropy Calculation

To compute entropy without human supervision, traces (hierarchical span trees) are converted into deterministic textual representations. This process involves:

Ordering: Sorting spans hierarchically and temporally.
Noise Filtering: Removing high-cardinality attributes (e.g., dynamic IPs, ports) irrelevant to behavioral structure.
Quantization: Discretizing continuous values (e.g., latency) into buckets to prevent natural network jitter from artificially inflating entropy.

3. Key Contributions

The paper identifies three primary contributions:

The RADAR Framework: A practical implementation of an RL agent integrated into OpenTelemetry that adjusts sampling policies at runtime without requiring manual intervention or prior data labeling.
Entropy-Based Reward Metric: A methodology for quantifying the relevance of distributed traces. It prioritizes traces with greater informational diversity (high entropy) and discards redundant requests, effectively balancing operational cost with information value.
Experimental Microservices Environment: The creation of "Minimal Boutique," a controllable, containerized testbed based on the Google Online Boutique architecture. It features fault injection and stochastic load generation to evaluate agent convergence and system performance.

4. Experimental Results

Experiments were conducted on a Kubernetes cluster simulating an online store, comparing RADAR against a full collection baseline (100% sampling) and a fixed-rate baseline (20% sampling).

4.1 Resource Efficiency

Bandwidth: RADAR reduced network bandwidth consumption by 97.4% compared to full collection (dropping from ~344 Kbit/s to ~9 Kbit/s).
CPU Usage: RADAR reduced CPU usage by 99.0% compared to full collection (dropping from ~1.99% to ~0.02%).
Memory: RADAR consumed 58.0% less memory than the full collector.
Comparison to Fixed Sampling: RADAR outperformed a fixed 20% sampling rate, achieving similar or better resource savings while maintaining higher data quality.

4.2 Observability and Data Quality

Entropy Increase: The average entropy of the stored information increased by approximately 25% (from 5.76 to 7.63), indicating a shift toward capturing more diverse and informative traces.
Rare Trace Preservation: Crucially, RADAR retained approximately 85.6% of rare trace patterns (traces appearing only once) identified in the full baseline, despite collecting only ~13% of the total trace volume.
Convergence: The agent demonstrated stable convergence to optimal policies regardless of the number of available rules (tested with 3, 6, 9, and 12 rules), maximizing the reward function over time.

5. Significance and Claims

The paper claims that RADAR validates the feasibility of using entropy to orchestrate telemetry autonomously and efficiently. Its significance lies in:

Proactive vs. Reactive: Unlike reactive systems (e.g., Hindsight) that rely on predefined anomaly triggers, RADAR proactively discovers relevant traces through entropy maximization, capturing scenarios that might not trigger static rules.
Portability: By operating entirely in user space via OpenTelemetry, RADAR avoids the complexity and privilege requirements of kernel-level instrumentation (e.g., eBPF) or invasive compiler modifications.
Dynamic Balance: The approach offers a superior dynamic balance between operational cost and informational value compared to manual tail-sampling configurations, which are difficult to maintain in dynamic cloud environments.

6. Limitations and Future Work

The authors acknowledge several limitations:

External Validity: Experiments were conducted solely on "Minimal Boutique," a custom environment. Performance on more complex topologies or unforeseen workloads remains to be validated.
Baseline Comparison: The study did not compare RADAR against a static configuration using the same rule catalog with fixed probabilities, which could better isolate the value of the learned policy.
Hyperparameter Sensitivity: Reward function parameters were selected via manual experimentation rather than systematic sensitivity analysis.
Operational Overhead: The cost of the Kubernetes rolling updates triggered by policy changes was not quantified.

Future work directions include evolving rule discarding to probabilistic activation rates, expanding the model to a full MDP by incorporating cluster state, integrating business metrics into the reward function, and validating the approach on independent industry benchmarks.