⚡ electrical engineering

Risk-Aware AoII-Based Scheduling with Hybrid Transmission for a Semi-Markov Source

This paper proposes and evaluates model-based and model-free scheduling policies for a multi-receiver system with a semi-Markov source that minimize a risk-aware cost function combining Age of Incorrect Information, risk ratio, and transmission costs by dynamically selecting between idling, unicast, and broadcast transmissions.

Original authors: Saeid Sadeghi Vilni, Risto Wichman

Published 2026-06-11

📖 4 min read☕ Coffee break read

CC0 1.0

Original authors: Saeid Sadeghi Vilni, Risto Wichman

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a busy traffic control center trying to keep a fleet of delivery trucks updated on road conditions. The center (the transmitter) watches a specific road segment that can be in different states: Free Flow, Congestion, or an Accident.

Here is the twist: The road doesn't change states randomly like a coin flip. Sometimes, a traffic jam lasts for exactly 5 minutes. Other times, it might drag on for 20 minutes or suddenly clear up in 1 minute. This unpredictable "staying power" of each state is what the paper calls a Semi-Markov source. It's like a weather system that knows it's been raining for an hour and is more likely to keep raining than if it had just started.

The center needs to send updates to multiple trucks (the receivers) driving on that road. The trucks need to know the current state to drive safely. If a truck thinks the road is clear but it's actually an accident, that's a problem. The paper measures this problem not just by "how old" the information is, but by Age of Incorrect Information (AoII). Think of AoII as a timer that starts ticking the moment a truck's map is wrong. The longer the timer runs, the more dangerous the situation becomes.

The Dilemma: Who Gets the Message?

The center has three choices every minute:

Stay Silent: Save energy, but the trucks might drive blind.
Unicast (The Private Call): Send a high-quality, targeted message to just one truck. This is like using a strong, focused spotlight. It's very reliable for that one truck, but the others might miss it.
Broadcast (The Megaphone): Shout the update to all trucks at once. This is less reliable (the signal might be weak for some), but it reaches everyone.

The goal of the paper is to figure out the perfect strategy: When should we whisper to one truck, shout to all, or stay quiet?

The "Risk" Factor

The authors realized that not all mistakes are equal. If a truck has been wrong for a little while, it's annoying. But if it has been wrong for a long time, it's risky.

They introduced a "Risk Alarm." If a truck's "wrongness timer" (AoII) gets too high, it enters a Risk State. The system's new goal is to minimize three things combined:

How wrong the trucks are on average.
How often trucks hit that "Risk Alarm."
The cost of sending messages (battery/energy).

The Solution: Two Brains, One Goal

To solve this, the authors built two types of "brains" (algorithms) to make the decisions:

The Perfect Planner (Model-Based): This brain knows the exact rules of the road (how long jams usually last) and the exact quality of the radio signals. It uses math to calculate the perfect move every single time.
The Learner (Model-Free/DQN): This brain doesn't know the rules beforehand. It learns by trial and error, like a video game player. It tries different strategies, sees what happens, and gets better over time.

They compared these smart brains against two "dumb" strategies:

The Panic Button: Always shout to the truck that has been wrong the longest.
The Threshold Shouter: Only shout to everyone if too many trucks are in the "Risk State."

What They Found

The results were clear:

The Smart Brains Win: Both the Perfect Planner and the Learner did much better than the dumb strategies. They knew exactly when to whisper to a critical truck and when to shout to the whole group.
Risk Matters: When the authors told the system to care more about avoiding "Risk States" (by turning up the "Risk Weight"), the smart brains changed their behavior. They became more aggressive in helping the trucks that were in danger, successfully keeping the "Risk Alarm" from going off.
The "Staying Power" Matters: The paper showed that it's not just how long a traffic jam lasts on average that matters, but how it lasts. A jam that lasts exactly 5 minutes every time is different from a jam that varies wildly between 1 and 10 minutes. The smart scheduling system needed to understand these patterns to work well.

The Takeaway

In simple terms, this paper teaches us that in a world where information needs to be fresh and correct, you can't just send updates randomly. You need a smart system that understands how long a situation is likely to last, who is in the most danger, and whether to talk to one person or the whole crowd. By using these smart strategies, you can keep everyone safe and informed without wasting energy.

Technical Summary: Risk-Aware AoII-Based Scheduling with Hybrid Transmission for a Semi-Markov Source

Problem Formulation
This work addresses a multi-receiver status update system where a transmitter monitors a finite-state semi-Markov source and communicates over error-prone wireless channels. The system involves $K$ receivers, each requiring real-time tracking of the source state. The core challenge lies in the transmitter's decision-making at each time slot: whether to remain idle, unicast an update to a single selected receiver, or broadcast a common update to all receivers.

Unlike traditional Age of Information (AoI) metrics which measure freshness, this study focuses on the Age of Incorrect Information (AoII), which quantifies the duration a receiver has operated with an incorrect estimate of the source state. The system model incorporates a semi-Markov source, where the dwell time in each state follows a general distribution, meaning future evolution depends on both the current state and the elapsed time since the last transition.

The objective is to minimize the long-term average sum of three components:

The average AoII across all receivers.
The average risk ratio, defined as the fraction of receivers whose AoII exceeds a prescribed threshold $\Delta_{th}$ .
The transmission cost associated with channel usage.

Methodology
The authors formulate the problem as a finite-state average-cost Markov Decision Process (MDP). The state space includes the augmented source state (current state and elapsed dwell time), the receiver-side decoded information (tracked via ACK feedback), and the AoII values for each receiver.

Two distinct policy approaches are developed to solve the MDP:

Model-Based Policy (SSP): Utilizing the Relative Value Iteration Algorithm (RVIA), this approach assumes full knowledge of the source dynamics and channel statistics. It computes a stationary scheduling policy that optimally maps states to actions (idle, unicast, or broadcast).
Model-Free Policy (LSP): For scenarios where system dynamics are unknown, a Deep Q-Network (DQN) is employed. This learning-based policy takes the system state as input and learns the action-value function to approximate the optimal scheduling decisions without explicit transition models.

These proposed policies are benchmarked against two heuristic baselines:

Maximum-AoII Unicast Policy (MAUP): Always unicast to the receiver with the highest AoII.
Threshold-Based Broadcast Policy (TBP): Broadcasts if the number of receivers in a "risk state" (AoII > $\Delta_{th}$ ) exceeds a threshold $m$ ; otherwise, it unicasts to the receiver with the highest AoII.

Key Results
Numerical simulations demonstrate the following findings:

Performance Superiority: Both the SSP and the learned LSP significantly outperform the MAUP and TBP baselines in terms of average cost and average AoII. The LSP successfully captures the scheduling tradeoffs without requiring explicit knowledge of the transition model.
Risk Management: Increasing the risk-weight parameter $\gamma$ in the cost function allows the proposed policies to adaptively reduce the average risk ratio. The SSP achieves the lowest risk ratio, while the LSP approaches this performance, confirming that the risk-aware formulation effectively controls the occurrence of high-AoII states.
Hybrid Transmission Utility: The learned policy dynamically exploits both unicast and broadcast modes. As the number of receivers increases, the scheduler becomes more aggressive in channel usage. The choice between unicast (higher success probability for a specific receiver) and broadcast (lower success probability but benefits multiple receivers) is optimized based on the reliability tradeoff and the number of receivers in risk states.
Impact of Dwell-Time Law: The study compares three different dwell-time distributions (geometric, deterministic, and truncated power-law) with identical mean dwell times. Results indicate that the specific dwell-time law significantly affects scheduling performance, even when the mean is fixed. The proposed semi-Markov formulation captures these nuances, whereas models assuming specific dwell-time laws (like standard Markov) would miss this dependency.

Significance and Claims
The paper claims that its primary contribution is the formulation of a risk-aware scheduling problem that moves beyond simple update freshness (AoI) to consider the correctness of receiver estimates and the operational risk of prolonged incorrectness. By integrating the semi-Markov source dynamics directly into the scheduling state, the proposed policies account for the elapsed dwell time, which is critical for physical processes where state persistence varies.

The work demonstrates that adaptive hybrid transmission (switching between unicast and broadcast) driven by AoII and risk metrics yields superior performance compared to fixed or purely AoI-driven strategies. Furthermore, the results highlight that the underlying statistical law of the source's dwell time is a critical factor in system performance, validating the necessity of using semi-Markov models over simpler Markov approximations for such real-time tracking applications.