Technical Summary: Vision-Language Models on the Edge for Real-Time Robotic Perception

Problem Statement

Vision-Language Models (VLMs) offer significant potential for robotic perception and interaction by enabling multimodal reasoning, such as inferring human intentions and decomposing complex environments. However, their deployment in real-world robotic systems faces three primary constraints:

1. Latency: Cloud-based inference introduces excessive wide-area latency, conflicting with the millisecond-to-second response times required for safe navigation and interaction.
2. Resource Constraints: Mobile and humanoid robots have strict size, weight, and power budgets, limiting onboard access to high-end GPUs and large memory required for state-of-the-art models.
3. Privacy and Bandwidth: Offloading raw multimodal streams (video, audio, LiDAR) to remote servers consumes significant bandwidth and raises privacy risks, while relying on stable connectivity that may be unavailable in dynamic environments.

While 6G-enabled edge intelligence, specifically Open RAN (ORAN) and Multi-access Edge Computing (MEC), offers a pathway to bring computation closer to the data source, there is a lack of empirical evidence regarding the performance of large VLMs on such infrastructure in real robotic systems. Existing studies often rely on simulations or virtualized nodes, leaving critical questions about latency–accuracy trade-offs unresolved.

Methodology

The authors propose and evaluate a real-time robotic perception pipeline deployed on an ORAN/MEC edge node using the Unitree G1 humanoid robot as an embodied testbed.

System Architecture

The system integrates three core components:

• Embodied Platform (Unitree G1): Functions as the User Equipment (UE) and primary data source. It utilizes a four-microphone array for speech input (processed via ASR) and an Intel RealSense D435i camera for synchronized RGB and depth streams.
• Communication Layer: A WebRTC-based pipeline facilitates low-latency, peer-to-peer communication. Video streams are transmitted via SRTP, while text queries and control metadata are exchanged via encrypted WebRTC data channels. A signaling server manages session handshakes (SDP/ICE).
• Edge Compute Node: Hosts the VLM inference service using FastAPI. The hardware configuration includes an NVIDIA L4 GPU (24 GB VRAM), reflecting a realistic ORAN/MEC node profile.

Model Configuration

Two models were evaluated to analyze the trade-off between reasoning capability and responsiveness:

1. LLaMA-3.2-11B-Vision-Instruct: A large-scale model deployed with 4-bit NF4 quantization (double quantization) to fit edge memory constraints. Inference is optimized with greedy decoding, a 50-token limit, and early stopping.
2. Qwen2-VL-2B-Instruct: A compact 2B parameter model optimized for resource-constrained environments, deployed on the same edge node to serve as a lightweight baseline.

Experimental Setup

The evaluation compared three deployment scenarios:

• Edge (Local): Both models running on the ORAN/MEC node.
• Cloud Baseline: LLaMA-3.2 deployed via the NVIDIA NIM cloud API, using identical prompts and preprocessing.
• Datasets:
  • Robo2VLM Benchmark: A large-scale dataset for VQA in robotic manipulation.
  • Robot-Collected Dataset: 200 Q&A pairs gathered in a laboratory environment covering five domains: navigation, instruction grounding, gesture recognition, human presence/activity, and human-robot interaction.

Performance was measured using End-to-End (E2E) Latency (from frame capture to response receipt) and Accuracy (exact match for binary/multiple-choice; normalized string match for free-form).

Key Contributions

1. System Integration: The authors designed and implemented a real-time pipeline deploying a large-scale VLM on an ORAN/MEC edge node, demonstrating the feasibility of multimodal inference in a realistic robotic setting.
2. Empirical Evaluation: A systematic comparison of edge versus cloud deployment for LLaMA-3.2 was conducted using both standardized benchmarks and real-world robot data, quantifying the specific impacts of deployment location on latency and accuracy.
3. Latency–Accuracy Trade-off Analysis: The study evaluated a compact model (Qwen2) on the edge, highlighting how lightweight architectures achieve sub-second responsiveness at the cost of reasoning depth, thereby illustrating design trade-offs for latency-critical interactions.

Results

• Edge vs. Cloud (LLaMA-3.2): Edge deployment preserved near-cloud accuracy while reducing E2E latency by 5% (1600.03 ms on edge vs. 1685.20 ms on cloud). While the absolute latency gain was modest, the relative improvement contributed to a higher accuracy-per-millisecond ratio, pushing responsiveness closer to sub-second thresholds.
• Model Comparison:
  • LLaMA-3.2: Achieved 41% accuracy on the Robo2VLM benchmark and demonstrated balanced performance across all human-robot interaction domains on the robot-collected dataset.
  • Qwen2-VL-2B: Achieved 28.02% accuracy on the benchmark (approx. 13% lower than LLaMA) but reduced latency to less than half of the cloud baseline, achieving sub-second responsiveness. It performed competitively on perception-driven tasks (e.g., human presence detection) but lagged in reasoning-intensive domains (e.g., spatial relations).
• Latency Breakdown: Analysis of the local LLaMA deployment revealed that text generation (autoregressive decoding) accounted for over 85% of the total E2E latency, while image decoding and preprocessing were negligible. This identifies the generation stage as the primary bottleneck.

Significance and Claims

The paper claims that deploying VLMs on ORAN/MEC infrastructure is a viable platform for multimodal AI in robotics. The significance of the work lies in:

• Feasibility Demonstration: Proving that large-scale VLMs can retain reasoning performance when migrated from cloud servers to resource-constrained edge hardware.
• Trade-off Clarity: Establishing a clear Pareto frontier where large models (LLaMA) offer performance optimality and compact models (Qwen2) offer efficiency optimality.
• System-Level Benefits: Beyond latency, edge deployment mitigates privacy risks and reduces bandwidth consumption by avoiding the offloading of raw sensory data.
• Future Direction: The authors suggest that future work should focus on model-system co-design to address the autoregressive decoding bottleneck and the development of adaptive hybrid strategies, where lightweight models handle fast perceptual grounding while larger models are selectively invoked for complex reasoning.