Goal-Oriented Status Updating for Real-time Remote Inference over Networks with Two-Way Delay

This paper proposes a goal-oriented scheduling framework for real-time remote inference over networks with two-way delays, modeling the problem as a Semi-Markov Decision Process to derive index-based threshold policies that jointly optimize packet freshness, length, and transmission timing, thereby significantly reducing inference error compared to traditional age-based scheduling.

The Gist

Imagine you are the captain of a spaceship trying to steer a robot on Mars. You can't see the robot directly; you only see a video feed sent back to Earth. But there's a catch: the signal takes time to travel, and sometimes the internet is fast, sometimes it's slow, and sometimes it gets stuck in traffic.

This paper is about how to send the best possible "instructions" to that robot when the connection is unreliable.

The Problem: The "Stale Sandwich" Dilemma

Usually, when we send data, we think: "Fresh is always better." If I send you a sandwich, you want the one made 5 minutes ago, not the one made 5 hours ago. In the world of data, this is called Age of Information (AoI). The older the data, the worse the prediction.

But here is the twist: Sometimes, sending a "stale" sandwich is actually better!

Imagine you are trying to predict the weather. If you send a report from right now, it might be chaotic and hard to read. But if you send a report from exactly 24 hours ago, it might perfectly match the pattern of today's weather because the weather is cyclical. In this case, the "old" data is actually more useful than the "fresh" data.

Furthermore, you have to decide how big the package is.

• Small package (1 sample): Fast to send, but might not have enough info to make a good guess.
• Huge package (100 samples): Takes a long time to send. By the time it arrives, it might be so old that it's useless, even though it had lots of data inside.

The Solution: The Smart Mailman

The authors of this paper designed a "Smart Mailman" (a scheduler) who decides three things for every package:

1. When to send it: Should I wait a few seconds for a better connection, or send it now?
2. How big the package should be: Do I send a quick text (small packet) or a heavy video file (large packet)?
3. Which data to pick: Do I send the newest data, or do I dig into the buffer and send an older piece of data that fits a specific pattern better?

The "Traffic Light" Analogy

Think of the network delay (how long it takes to send a message) like a traffic light that changes colors.

• Green Light (Fast): The road is clear.
• Red Light (Slow): The road is jammed.

In the past, people assumed the traffic lights changed randomly, like flipping a coin. But in reality, if the light is red now, it's likely to stay red for a bit (this is called Markovian delay or "memory").

The authors' new system is like a smart driver who knows the traffic pattern.

• If the light is about to turn green, the driver waits a split second to send a big package (because it will get through fast).
• If the light is stuck on red, the driver sends a tiny package immediately, or maybe waits for a specific "stale" piece of data that is known to work well even with delays.

The Two Strategies

The paper offers two ways to drive this smart system:

1. The "Set It and Forget It" Strategy (Time-Invariant)
You decide once: "I will always send packages of size 5." The system then just figures out when to send them and which data to pick. This is simple and works great if your computer isn't very powerful.

2. The "Dynamic Driver" Strategy (Time-Variable)
The system changes its mind constantly. "Right now, the traffic is bad, so I'll send a tiny package. Five minutes later, the traffic is great, so I'll send a huge package." This is the most powerful method but requires a very smart computer to calculate the best move every second.

The Results: Why It Matters

The authors tested this on two scenarios:

1. Math Problems: Predicting a mathematical pattern (like a bouncing ball).
2. Robot Control: Predicting the position of a "Cart-Pole" (a classic balancing robot).

The Magic Number:
When they compared their "Smart Mailman" to the old way of doing things (just sending the freshest, smallest data possible), the results were shocking.

• The old way made mistakes.
• The new way reduced those mistakes by six times (dropping the error down to one-sixth).

The Takeaway

In a world where we rely on AI to control self-driving cars, robots, and digital twins of space stations, we can't just shout "Send the newest data!" into the void. We need to be strategic.

Sometimes, waiting is better. Sometimes, sending an older piece of data is smarter. And sometimes, sending a bigger package is worth the wait. This paper gives us the mathematical recipe to know exactly when to do what, ensuring our AI gets the right information at the right time, even when the internet is acting up.

Technical Summary

Here is a detailed technical summary of the paper "Goal-Oriented Status Updating for Real-time Remote Inference over Networks with Two-Way Delay."

1. Problem Statement

The paper addresses the challenge of real-time remote inference in next-generation networks where an intelligent model (e.g., a pre-trained neural network) at a receiver must infer the real-time value of a target signal ($Y_t$) using data samples transmitted from a remote source.

Key constraints and characteristics of the problem include:

• Goal-Oriented Communication: The objective is not merely to transmit data reliably or maximize throughput, but to minimize the inference error at the destination.
• Non-Monotonic Age Dependence: Unlike traditional Age of Information (AoI) studies which assume performance degrades monotonically as data ages, this paper acknowledges that inference error can be non-monotonic with respect to AoI. In some scenarios (e.g., periodic signals or delayed relationships between source and target), slightly "stale" data may yield better inference results than the freshest possible data.
• Packet Length vs. Freshness Trade-off: The scheduler must decide the packet length ($l$, number of samples) and the freshness (which samples from the buffer to send). Longer packets provide more information (reducing error) but incur longer transmission delays, increasing the AoI of the data upon arrival.
• Two-Way Delay with Memory: The system operates under two-way delays (forward transmission delay and feedback delay) that are not independent and identically distributed (i.i.d.). Instead, the delay states evolve according to a Markov chain, meaning the current network condition depends on the previous state (delay memory).
• Selection-from-Buffer Model: The scheduler can choose which samples to transmit from a buffer of recent samples, rather than being forced to generate new data immediately ("generate-at-will").

2. Methodology

The authors model the system as an infinite-horizon average-cost Semi-Markov Decision Process (SMDP). The system state includes the current AoI, the length of the last delivered packet, and the current network delay state.

The paper tackles two distinct optimization scenarios:

A. Time-Invariant Packet Length Selection

In this scenario, the packet length ($l$) is fixed for all transmissions.

• Formulation: The problem is decomposed into a two-layer nested optimization:
  1. Inner Layer: For a fixed $l$, optimize the waiting time (when to transmit) and the buffer position (which samples to pick). This is modeled as an SMDP.
  2. Outer Layer: Optimize the constant packet length $l$ by searching over possible integer values.
• Solution: The authors derive a closed-form solution for the inner layer.
  • Waiting Time: Determined by an index-based threshold policy. The scheduler waits until an index function (dependent on current AoI and delay state) exceeds a threshold ($\beta$).
  • Buffer Position: Determined by a stationary function of the delay state only (independent of AoI).
  • Complexity: This approach avoids the high computational cost of standard dynamic programming by providing explicit analytical expressions.

B. Time-Variable Packet Length Selection

In this scenario, the packet length can change dynamically based on the system state.

• Formulation: Modeled directly as a single SMDP where actions include waiting time, packet length, and buffer position.
• Challenge: Solving the standard Bellman optimality equation for this joint optimization is computationally expensive ($O(B^3)$ complexity).
• Solution: The authors derive a structural result showing that the optimal waiting time can be decoupled from the packet length decision.
  • They propose a simplified Bellman optimality equation where the waiting time is determined by a threshold rule (similar to the time-invariant case), and the packet length/buffer position are optimized separately.
  • This reduces the time complexity significantly (from $O(B^3)$ to roughly $O(B^2)$ or $O(B)$ depending on implementation), making real-time implementation feasible.

3. Key Contributions

1. Generalized System Model: Extends previous remote inference models to include Markovian two-way delays (capturing delay memory) and non-monotonic inference error functions.
2. Closed-Form Optimal Policies:
   • Derives a closed-form solution for the time-invariant packet length case, proving that the optimal buffer selection depends only on the network delay state, while the waiting time depends on both AoI and delay state.
   • Develops an index-based threshold policy for both fixed and variable packet lengths.
3. Complexity Reduction: For the time-variable case, the authors derive a simplified Bellman equation that drastically reduces computational complexity compared to solving the full joint optimization problem, enabling practical deployment.
4. Co-Design Framework: Successfully integrates learning (inference model performance) and communication (scheduling/delay) into a unified optimization framework.

4. Simulation Results

The authors evaluate their policies using two experiments:

1. Model-Based (AR Process): Remote inference of an Auto-Regressive process.
   • Result: The proposed goal-oriented scheduler reduces the time-average inference error to one-sixth of the error achieved by a standard "generate-at-will, zero-wait, unit-length" policy.
   • Delay Memory: The policy utilizing delay memory (Markovian) outperforms policies assuming i.i.d. delays by up to 11.6% when delay memory is high.
2. Trace-Driven (Cart-Pole): Predicting the state of a cart-pole system using video frames.
   • Result: The time-variable packet length policy outperforms constant-length policies by up to 15.7%, demonstrating the value of dynamically adjusting packet size based on network conditions.
   • Complexity: The simplified Bellman equation reduces computation time by more than two-fold for larger buffer sizes compared to the original equation.

5. Significance

This paper is significant for the design of Goal-Oriented Networking for AI applications (e.g., digital twins, autonomous vehicles, industrial robotics).

• Paradigm Shift: It challenges the conventional wisdom that "freshness" (lowest AoI) is always optimal. It proves that in systems with non-monotonic error functions, selecting slightly older data or adjusting packet sizes can significantly improve task performance.
• Practicality: By accounting for delay memory (Markovian delays), the proposed policies are more robust for real-world networks (like satellite or 5G/6G) where congestion and routing conditions persist over time.
• Efficiency: The derivation of closed-form solutions and complexity-reduced algorithms makes these theoretically optimal policies implementable in real-time systems, bridging the gap between theoretical control theory and practical network engineering.