🤖 AI

RAMS: Resource-Adaptive and Detection-Conditioned Model Switching for Embedded Edge Perception

RAMS is a lightweight, resource-adaptive runtime controller that dynamically switches between YOLOv8 model tiers based on device pressure and recent vulnerable-road-user detections to optimize inference latency and detection quality on embedded edge hardware without model-reload overhead.

Original authors: Kushal Khemani, Evan Leri, George Xu, Amit Hod

Published 2026-06-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Kushal Khemani, Evan Leri, George Xu, Amit Hod

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are driving a car with a very smart, but slightly nervous, co-pilot. This co-pilot's job is to watch the road for pedestrians, cyclists, and other vulnerable people (what the paper calls "VRUs") and shout out warnings.

The problem is that the car's computer (the "brain") sometimes gets overwhelmed. Maybe the radio is blasting, the GPS is calculating a new route, and the engine is running hot. When the computer is stressed, it gets slow. If the co-pilot tries to use a super-detailed, high-tech map to spot a pedestrian, the car might freeze for a second while it processes the image. That's dangerous. But if the co-pilot switches to a super-fast, blurry map to save time, they might miss a pedestrian entirely.

RAMS is the name of a new system designed to solve this exact problem. It acts like a smart traffic manager for the car's computer, constantly deciding which "map" (or AI model) to use based on how busy the computer is and what it just saw.

Here is how it works, broken down into simple concepts:

1. The Three "Maps" (Model Tiers)

RAMS doesn't just have one way of seeing the world; it has three versions of the same AI, all loaded and ready to go at the same time:

The "NANO" Map: Super fast, very low detail. It's like looking at the road through a tiny keyhole. It's great for speed but might miss a small child standing far away.
The "SMALL" Map: A middle ground. Good balance of speed and detail.
The "MEDIUM" Map: High detail, slow speed. It's like looking through a high-definition telescope. It sees everything clearly but takes a long time to process.

2. The "Pressure Gauge" (Resource Monitor)

RAMS constantly checks the computer's "stress level." It looks at how much CPU power is being used, how hot the computer is getting, and how much memory is free.

The Analogy: Imagine a gauge on your dashboard. If the needle is in the green (low stress), RAMS says, "We have plenty of power, let's use the high-definition MEDIUM map to be safe."
If the needle swings to the red (high stress), RAMS says, "We are overloaded! Switch to the fast NANO map immediately so we don't freeze."

3. The "Smart Switch" (No Restart Needed)

In older systems, switching from a slow map to a fast one often required reloading the software, which caused a lag (like restarting your phone every time you wanted to change apps).
RAMS keeps all three maps loaded in memory at the same time (called "Warm Tiers"). Switching between them is instant, like flipping a light switch. There is no lag, no waiting, and no "reloading."

4. The "Safety Lock" (Detection-Conditioned Switching)

This is the paper's most clever feature. Usually, if the computer gets stressed, RAMS would immediately switch to the fast, blurry NANO map. But what if the computer just saw a pedestrian? Switching to the blurry map right then could be fatal.

RAMS has a special rule: If a vulnerable person is detected, the system refuses to switch to the blurry map for a few seconds.

The Analogy: Imagine you are walking in the dark. If you hear a twig snap (a detection), you don't immediately put on sunglasses (switch to the fast/blurry mode) even if you are tired. You keep your eyes wide open (stay on the detailed map) for a moment to make sure you don't trip.
RAMS uses the size of the detected object to guess how close it is. If a pedestrian looks big (close), it locks the system onto the high-detail map. If they look small (far away), it might allow a switch to the middle map.

5. The "Universal Translator" (Calibration)

One of the paper's big claims is that this system works on any computer, from a tiny Raspberry Pi to a powerful laptop, without needing a human to tweak the settings for each one.

The Analogy: Think of it like a thermostat that learns your house's "normal" temperature. When you first turn it on, it measures the room while nothing is happening (idle). It then sets its "too hot" and "too cold" limits based on that baseline.
RAMS does this automatically. It measures the computer's stress when it's doing nothing, then sets its switching rules relative to that baseline. This means the same code works perfectly on a slow device and a fast device without manual adjustments.

6. The "Scorecard" (SWAS)

How do you know if this system is actually safer? The authors created a new scoring method called SWAS.

Instead of just counting how many objects were found, SWAS gives extra points for finding vulnerable people (like pedestrians) when the computer was under stress.
They also created a "truth test" (Oracle) to make sure the system wasn't just tricking itself. They found that while the system is very good, it is limited by how good the "blurry" map is at seeing people in the first place. If the blurry map misses 76% of the pedestrians, the "Safety Lock" can't save you 76% of the time because it doesn't know the danger is there yet.

The Bottom Line

The paper shows that on a powerful computer (NVIDIA Jetson Orin), this system can run 5.6 times faster than a system that stubbornly sticks to the high-definition map, while still keeping 74% of the safety accuracy.

It achieves this by being flexible: it slows down when the computer is busy, but it smartly "locks" the safety features on whenever it thinks a person is in danger, ensuring that speed never comes at the cost of missing a life-threatening situation.

Technical Summary: RAMS – Resource-Adaptive and Detection-Conditioned Model Switching for Embedded Edge Perception

1. Problem Statement

Deploying object detection on embedded hardware faces a fundamental tension between inference latency and detection quality that is absent in cloud environments. While cloud inference latency is bounded primarily by model complexity and network round-trip time, embedded devices share CPU and memory resources with the host OS, background services, and sensor pipelines. As system utilization rises, inference latency for a fixed model degrades substantially (e.g., a YOLOv8-MEDIUM inference on an Intel i7-1165G7 can degrade from 448 ms at idle to over 1500 ms under burst load).

The standard trade-off—downgrading to a lighter model to reduce latency—introduces an asymmetric cost. A 320 px inference on an empty road incurs the same accuracy penalty as one containing a nearby pedestrian, yet missing a pedestrian carries a safety risk not captured by latency metrics alone. Furthermore, existing adaptive systems often rely on fixed thresholds that do not transfer across heterogeneous hardware due to varying idle resource pressures, or they require cloud connectivity, which is unavailable on edge devices.

2. Methodology: The RAMS System

RAMS (Resource-Adaptive and Detection-Conditioned Model Switching) is a lightweight runtime controller designed to balance latency and safety without model-reload latency. It operates entirely on-device and consists of four components:

A. Resource Pressure Monitor

Running at 10 Hz, the monitor queries OS-level signals (CPU utilization, virtual memory pressure, temperature, and battery charge) to compute a scalar pressure index $R(t) \in [0, 1]$ .
$R(t) = w_c c + w_m m + w_T \hat{T} + w_b (1 - b)$
The weights reflect the empirical dominance of CPU utilization in embedded workloads. If specific sensors are unavailable, their weights are redistributed to CPU utilization.

B. Idle-Relative Threshold Calibration

To ensure portability across devices without per-device hand-tuning, RAMS employs an idle-relative calibration procedure. Instead of using fixed thresholds, the system profiles the device at idle (no inference load) to determine a baseline pressure $R_{idle}$ . Switching thresholds are then set as fixed offsets from this baseline:
$\theta_\ell = R_{idle} + \delta_\ell, \quad \theta_h = R_{idle} + \delta_h$
where $\delta_\ell = 0.10$ and $\delta_h = 0.25$ . This ensures that the semantic meaning of the thresholds (e.g., "moderate load") remains consistent across platforms with different idle baselines (e.g., Raspberry Pi 5 vs. Jetson Orin).

C. Warm Model Library

RAMS maintains three YOLOv8 tiers (NANO at 320px, SMALL at 416px, MEDIUM at 640px) resident in memory simultaneously. This "warm-tier" approach eliminates the latency penalty associated with loading models on demand. The total memory footprint is approximately 81 MB for ONNX FP32 models and 120–150 MB for TensorRT engines.

D. Switching Policies

The controller implements five policies ranging from simple thresholding to detection-conditioned overrides:

Threshold Policy: Uses hysteresis to prevent rapid oscillation between tiers based on $R(t)$ .
Predictive Policy: Uses an Exponentially Weighted Moving Average (EWMA) to forecast pressure and trigger downgrades before thresholds are crossed.
Adaptive Policy: Self-tunes the EWMA smoothing factor based on the variance of recent pressure readings.
Safety Policy: A detection-conditioned override that prevents downgrading to the NANO tier within a temporal window ( $\Delta t_w = 0.5$ s) after a Vulnerable Road User (VRU) is detected.
Two-Level Safety Policy (Safety2): An enhanced version that conditions the override tier on the bounding box area of the detected VRU. Large boxes (near objects) trigger a MEDIUM tier lock, while smaller boxes (distant objects) trigger a SMALL tier lock.

3. Key Contributions

Formal Resource-Pressure Index: A portable, idle-relative calibration method ( $R(t)$ ) that allows the same controller equations to operate across a 37× latency range on heterogeneous hardware without per-device tuning.
Detection-Conditioned Switching: Five formally specified policies, including two reactive variants that delay downgrades based on recent VRU detections, addressing the asymmetric cost of missing safety-critical objects.
SWAS Metric: The VRU-Weighted Accuracy Score (SWAS), a scalar metric for offline policy comparison on unlabeled video. It weights accuracy by VRU presence. The paper introduces an "oracle-bounded" variant ( $SWAS^\star$ ) that uses ground-truth labels to separate the benefits of tier retention from the circularity of using the detector's own (potentially imperfect) output to trigger the policy.
Comprehensive Evaluation: A ten-experiment evaluation across five hardware targets (Raspberry Pi 5, Intel i7-1165G7, i7-13700F, Jetson Orin ONNX, Jetson Orin TensorRT) covering load sensitivity, hysteresis tuning, and safety-latency trade-offs.

4. Results

Experiments were conducted using YOLOv8 tiers and the KITTI dataset. Key findings include:

Latency and Accuracy Trade-off: On Jetson Orin with TensorRT under heavy load, the Safety2 policy achieved a mean latency of 3.41 ms (5.6× faster than fixed-MEDIUM) while retaining 74% of the proxy accuracy. It achieved this by operating primarily in NANO mode but selectively locking to SMALL or MEDIUM during VRU-positive windows.
SWAS Improvements: Under heavy load, detection-conditioned switching improved SWAS by 25.4% under oracle scoring (the conservative estimate accounting for detector circularity) and 47.3% under detector-derived scoring compared to threshold-only policies.
Cross-Device Portability: The same controller equations governed operation across a 37× latency range (from 3.41 ms on Jetson TRT to 127.52 ms on Jetson ONNX). The system successfully adapted tier selection to the specific hardware capabilities without manual threshold adjustment.
Recall Bounds: The paper explicitly quantifies a fundamental limit of reactive systems. With the NANO tier achieving only 24.2% VRU recall on KITTI, the safety override is inactive for 76% of VRU-present frames because the base detector failed to trigger the condition. This highlights that reactive switching cannot compensate for the inherent recall limitations of the lowest-tier model.
Transient Response: The controller commits to tier changes within 3 monitor samples (0.3s) for threshold policies and faster for predictive/adaptive policies. Safety policies introduce a maximum of 5 additional cycles at a higher tier after a VRU event before reverting.

5. Significance and Claims

The paper claims that RAMS is the only system combining warm-tier switching, detection-conditioned tier selection, cross-device portable calibration, and full on-device operation without cloud offload.

The authors emphasize that the 25.4% oracle SWAS improvement represents a genuine benefit of detection-conditioned switching, distinct from the inflated gains seen in detector-derived scoring caused by the circularity of using a low-recall detector to trigger safety locks. The work demonstrates that it is possible to achieve near-NANO latency while preserving a significant portion of the accuracy of heavier tiers during critical safety windows.

However, the paper is modest regarding the limitations of this approach. It explicitly states that the reactive override is bounded by the recall of the base detector; if the NANO tier misses a VRU, the safety lock cannot fire. The authors identify this as a fundamental constraint of any reactive detection-conditioned system operating below an imperfect detector, suggesting that future work must focus on stronger lightweight baseline detectors or parallel VRU classifiers to close this gap. The system does not claim to provide formal safety guarantees (e.g., ISO 26262) but rather a mechanism to optimize the latency-accuracy trade-off in resource-constrained environments.