TURBO: Utility-Aware Bandwidth Allocation for Cloud-Augmented Autonomous Control

Technical Summary: TURBO: Utility-Aware Bandwidth Allocation for Cloud-Augmented Autonomous Control

1. Problem Statement

Modern Autonomous Vehicles (AVs) rely on increasingly large and complex Machine Learning (ML) models to ensure safety. However, a significant "compute gap" exists: state-of-the-art (SOTA) models require massive computational power (e.g., NVIDIA H100 GPUs) that is economically and physically infeasible to deploy entirely on-vehicle due to power, thermal, and cost constraints.

While the cloud offers abundant compute, the network has traditionally been treated as an unreliable bottleneck. Previous attempts to offload AV tasks to the cloud often failed because they ignored the bandwidth-induced latency. In cellular networks (like 5G), the time required to transmit high-resolution sensor data (often exceeding 8 Gbps) can exceed the strict real-time Service Level Objectives (SLOs) required for safe driving (e.g., 150–250ms), rendering cloud-based inference useless if the data cannot arrive in time.

2. Methodology

The authors propose TURBO (Task Utility and Resource Bandwidth Optimizer), a framework that treats bandwidth allocation and control pipeline configuration as a joint optimization problem. Instead of treating the network as a static pipe, TURBO dynamically reconfigures the entire pipeline based on real-time network conditions.

Key components of the methodology include:

• Model Families & Reconfigurability: TURBO leverages the fact that ML tasks (like object detection) have "families" of models. For any given task, the system can choose from a range of models with different tradeoffs between accuracy, input size, and execution time.
• Utility-Aware Modeling:
  • Model-Level Utility: The authors define a "step-function" utility for each model. A model provides high utility (its accuracy score) only if it can complete within the latency SLO; otherwise, its utility is zero.
  • Service-Level Utility: Each service (e.g., a specific camera stream) aggregates the utilities of its available models (on-vehicle vs. various cloud configurations) to create a service-level utility curve.
  • Application-Level Utility: The system combines individual service utilities into a global objective function, allowing for the prioritization of critical services (e.g., the front camera) over less critical ones.
• Optimization via ILP: At runtime, TURBO collects real-time estimates of available bandwidth and Round-Trip Time (RTT). It then solves an Integer Linear Program (ILP) to select the optimal combination of cloud models and bandwidth allocations that maximizes the total application-level utility without exceeding the available bandwidth.
• Dynamic Compression: To maximize the utility of limited bandwidth, TURBO considers multiple preprocessing and compression configurations (e.g., JPEG, PNG, or DNN-input compression) to find the best balance between data size and accuracy.

3. Key Contributions

• Co-design Framework: A novel approach that integrates control, ML model selection, and network resource allocation into a single optimization loop.
• Utility-Based Allocation: A mathematical framework that uses step-function utility curves to handle the discrete nature of model selection and the "all-or-nothing" requirement of real-time SLOs.
• End-to-End Implementation: A practical system implemented in Rust and Python, utilizing the QUIC protocol to manage concurrent data streams and provide real-time network telemetry (RTT and Congestion Window).

4. Results

The authors evaluated TURBO using both large-scale simulations (using Waymo Open Dataset traces) and real-world test drives.

• Accuracy Gains: In simulation, TURBO improved average accuracy by up to 15.6 percentage points (%pt) over on-vehicle-only pipelines and 12.7 %pt over naive bandwidth allocation methods.
• Efficiency of Design Choices: The study showed that the improvement comes from a synergy of three factors: having multiple model options (D1), using ILP for optimal distribution (D2), and applying dynamic compression (D3).
• Real-World Feasibility: During a 2-hour real-world test drive in a metropolitan area, TURBO successfully adapted to fluctuating 5G conditions, maintaining a fallback to on-vehicle models whenever network latency threatened the SLO, ensuring baseline safety was never compromised.
• Low Overhead: The ILP solver runs in approximately 14ms on edge hardware, making it suitable for real-time execution.

5. Significance

TURBO represents a paradigm shift in autonomous systems design. It moves away from the "edge vs. cloud" dichotomy toward a cloud-augmented model where the network is a first-class citizen in the control loop.

The implications are broad:

• For AV Manufacturers: It provides a path to deploy SOTA intelligence without the prohibitive cost of massive on-board hardware.
• For Mobile Networks: It highlights the growing demand for high-uplink bandwidth and low-latency guarantees to support safety-critical edge computing.
• Beyond AVs: The framework is applicable to any real-time, resource-constrained robotic system, such as autonomous drones or remote industrial robotics, where accuracy and latency are in constant tension.