💻 computer science

AgedNet-Motion: Training-Free Selective Task Refresh for Resource-Aware Streaming Multi-Task Perception

AgedNet-Motion introduces a training-free, resource-aware framework for streaming multi-task perception that reduces inference latency by selectively refreshing task-specific branches based on calibrated motion and age metrics while reusing cached outputs, achieving significant speedups with minimal quality loss across various models.

Original authors: Sungjun Jang, Taegeun Oh

Published 2026-10-19

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Sungjun Jang, Taegeun Oh

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of autonomous vehicles and mobile robots, seeing the world is not a single act but a constant, simultaneous conversation with the environment. A self-driving car must not only spot a pedestrian but also understand where the drivable road ends, where the lane markings begin, and how all these elements shift as the vehicle moves. To achieve this, engineers use systems that can perform multiple visual tasks at once, such as detecting objects and mapping the ground, all within a single computer program. These systems are designed to be efficient by sharing a common set of calculations for the initial steps of vision, much like a single person using one set of eyes to gather information before the brain decides how to interpret it for different needs. However, a persistent challenge remains: these systems often work too hard. They recalculate every single detail for every single frame of video, even when the scene outside the window has barely changed. This constant, repetitive computation drains battery power and slows down response times, which is dangerous when split-second decisions are required. The question facing researchers is whether these systems can learn to rest when the world is still, saving their energy for the moments when things actually move.

A team of researchers at Kyung Hee University and Dong Seoul University has developed a method called AgedNet-Motion to solve this problem without requiring the computer to learn anything new. Their approach is built on a simple observation: in a video stream, adjacent frames are often nearly identical. If a car is driving down a straight road, the image of the lane markings from one second ago is likely still accurate for the next second. Instead of forcing the computer to re-run the complex math for every task on every frame, the researchers designed a system that runs the shared, foundational part of the vision process on every frame but then selectively updates only the specific tasks that need it. The system keeps a memory of the results from previous frames, reusing them when they are still fresh, and only recalculating the parts of the vision that have become stale or when the scene changes significantly.

The core of this innovation is a decision-making process that is entirely free of training. Usually, to make a computer decide when to update its work, engineers would need to train a separate, complex model to predict when the current information is no longer good enough. This new method avoids that entirely. Instead, it uses a pre-calculated guide based on how long it has been since a task was last updated and how much the image has changed in the meantime. The system measures the difference between the current video frame and the previous one in grayscale, creating a simple score of motion. It then combines this motion score with a lookup table that tells it how much quality is lost when a specific task is left unupdated for a certain amount of time. If the scene is static, the system trusts the old results. If the scene is moving, it prioritizes refreshing the tasks that are most likely to be affected by that movement. This decision happens instantly, using only the current video feed and the stored memory of past results, requiring no additional learning or complex prediction models.

To test this idea, the researchers applied their method to three different existing multi-task vision models used for driving scenarios. They fed these models video streams from a large dataset of real-world driving footage and compared the performance of their selective system against the standard method, which updates everything on every frame. The results were striking. By skipping unnecessary calculations, the system reduced the time it took to process each frame by significant margins. For one of the models, the processing time dropped by more than forty-two percent; for another, the reduction was over fifty-one percent. Crucially, this speedup did not come at the cost of accuracy. The system maintained high-quality perception of the road, detecting vehicles and lane markings with a precision that was statistically indistinguishable from the full, slower system. In some cases, the selective method actually produced slightly better results than a simpler strategy that only considered how much time had passed, because it could react to sudden motion that a time-based clock would miss.

The researchers also verified that this method works in a real-world deployment setting, not just in a theoretical simulation. They exported the system to run on a standard graphics processor using a common industry software runtime, ensuring that the savings were not an artifact of a specific research setup. The performance gains held true, confirming that the system could deliver faster results on actual hardware without losing its ability to see the road clearly. The study demonstrated that the system is flexible enough to adapt to different types of vision models. For one model, the system learned that motion was a critical signal and used it heavily to decide when to update. For another model, the system determined that motion provided no benefit and automatically switched to a strategy based purely on time, effectively turning off the motion sensor for that specific task. This adaptability means the method can be applied to various existing systems without needing to retrain the underlying vision models or add new, heavy components.

The findings suggest a practical path forward for resource-constrained devices like self-driving cars and robots. By treating the freshness of information as a resource to be managed rather than a constant to be maintained, these systems can operate more efficiently. The AgedNet-Motion approach proves that it is possible to reduce the computational load of complex vision tasks by nearly half without sacrificing the quality of the output. It achieves this by intelligently deciding which parts of the visual world need to be re-examined and which can be trusted to remain the same, all without the need for additional training data or complex predictive algorithms. This work offers a clear, training-free solution to the problem of wasted computation in streaming video, allowing machines to see the world more efficiently while keeping their focus sharp on the moments that truly matter.

Technical Summary: AgedNet-Motion

Problem Statement
Conventional multi-task learning (MTL) models for streaming video perception, such as those used in autonomous driving (e.g., YOLOP, A-YOLOM), execute all task-specific branches on every video frame. While these models share a common feature-extraction backbone to avoid redundant low-level processing, they still recompute every task head (e.g., object detection, drivable area segmentation, lane marking) for every frame. This static execution pattern ignores temporal redundancy; adjacent frames often contain similar information, meaning that recomputing all outputs is computationally wasteful. Conversely, simply reusing old outputs without a smart strategy degrades perception quality as objects move or scenes change. Existing solutions often require training additional neural predictors or modifying the underlying MTL architecture, which introduces complexity and requires new training data. The challenge is to develop a resource-aware, selective execution system that reduces inference latency while maintaining output quality, without modifying the trained weights of existing MTL models or training a separate scheduler.

Methodology: AgedNet-Motion
The paper proposes AgedNet-Motion, a training-free, resource-aware selective refresh system. The approach operates on the following principles:

Architecture Separation: The system separates the existing MTL model into a common feature-extraction path ( $F$ ) and distinct task-specific branches ( $B_i$ ). The common path is executed on every frame, while only a single task branch is refreshed per frame after initialization.
Temporal Caching: Outputs for non-selected tasks are retrieved from a temporal cache using a "hold-last" policy. No optical flow or warping is applied; the system simply reuses the last computed output.
Training-Free Scheduling: The core innovation is a scheduler that decides which task to refresh based on two factors:
- Age-Regret Lookup Table: A task-specific table ( $L_i$ ) derived from offline calibration data. It maps the "age" of an output (frames since last refresh) to the expected "regret" (quality loss) of reusing that output. This is computed by comparing fresh outputs against ground truth at specific reference frames.
- Motion Signal: A cumulative grayscale frame difference ( $M_i(t)$ ) accumulated since the task was last refreshed. This captures the magnitude of scene changes.
Priority Score: The refresh priority for task $i$ at time $t$ is calculated as:
$S_i(t) = \max(L_i(\tilde{a}_i(t)), 0) \cdot (1 + w^* M_i(t))$
Where $\tilde{a}_i(t)$ is the capped age, and $w^*$ is a scalar motion weight.
Calibration Process: The system does not use gradient descent. Instead, it performs a grid search over a predefined set of motion weights ( $W = \{0, 1, 2, 4, 8, 16\}$ ) on a calibration partition of the dataset. It selects the weight $w^*$ that maximizes aggregate ground-truth quality across tasks. If $w^*=0$ , the motion term is disabled, effectively reverting to an "Age-Only" policy.
Overdue Guard: To prevent indefinite starvation of any task, a threshold $A_{max}$ is set. Once a task's age exceeds this threshold, it enters a "candidate set" where it competes for refresh regardless of its score.

Key Contributions

Resource-Aware Architecture: Implementation of a streaming architecture that decouples shared feature extraction from task-specific execution, allowing selective branch refresh without altering model weights or output structures.
Training-Free Scheduler: Proposal of AgedNet-Motion, which combines calibrated age-regret tables with a motion signal. It requires no additional predictor training, no gradient optimization, and no modification of the base MTL model.
Adaptive Calibration: Demonstration that calibration can automatically determine whether motion information is beneficial. For some models (e.g., A-YOLOM), the optimal weight is zero, reproducing a time-only policy; for others (e.g., YOLOP), it selects a high weight to leverage motion.
Comprehensive Evaluation: Evaluation across three distinct MTL models (YOLOP, A-YOLOM, YOLOPX) on BDD100K video streams, including deployment validation via ONNX Runtime on GPU.

Results
Evaluated on 200 held-out BDD100K videos, AgedNet-Motion achieved the following relative to full refresh (executing all branches every frame):

Latency Reduction:
- YOLOP: 42.19% reduction in mean model and post-processing latency.
- A-YOLOM: 51.54% reduction.
- YOLOPX: 33.24% reduction.
Quality Performance:
- YOLOP: Improved mean ground-truth (GT) quality by 0.453 percentage points over the "Age-Only" baseline (statistically significant, 95% CI > 0).
- A-YOLOM: Calibration selected $w^*=0$ , making the policy identical to Age-Only. This outperformed the "Tuned Fixed" (round-robin) baseline by 0.922 percentage points.
- YOLOPX: Outperformed Tuned Fixed by 0.940 percentage points. The difference from Age-Only was statistically indistinguishable from zero.
Deployment: When deployed via ONNX Runtime on an RTX 3060 GPU, the system preserved the relative latency reductions (43.2% for YOLOP, 52.3% for A-YOLOM) and maintained comparable accuracy to native PyTorch inference, confirming the benefits are not artifacts of the research implementation.

Significance and Claims
The paper claims that AgedNet-Motion provides a practical mechanism for controlling the resource-quality trade-off in existing streaming multi-task models. Its primary significance lies in its ability to:

Reduce Latency: Significantly lower inference latency by skipping redundant task computations without retraining the perception model.
Adapt Without Training: Automatically adapt its scheduling strategy (specifically, whether to use motion cues) through simple calibration, rather than requiring a learned predictor.
Robustness: Perform consistently across different MTL architectures, never being statistically confirmed worse than the strongest non-learned baseline (either Age-Only or Tuned Fixed) on any of the tested models.
Practicality: Function effectively in production environments (ONNX Runtime) without requiring complex dynamic computation graphs or additional neural networks.

The authors emphasize that the method is "training-free" in the sense that it does not train a new scheduler or modify the MTL weights; it only uses ground truth to build a lookup table and select a scalar weight during an offline calibration phase.

Technical Summary: AgedNet-Motion

More like this