Web-Gewu: A Browser-Based Interactive Playground for Robot Reinforcement Learning

1. Problem Statement

The paper addresses two critical barriers hindering the widespread adoption of robotics education and Reinforcement Learning (RL) research:

• High Computational Barriers: State-of-the-art RL simulation platforms (e.g., Isaac Lab, mjlab) require powerful local GPUs and complex environment configurations, excluding learners with limited hardware.
• Prohibitive Cloud Costs: Traditional centralized cloud simulation solutions offload computation to the server but incur massive GPU and bandwidth costs, making large-scale deployment economically unfeasible for universal education.
• The Accessibility Gap: Existing web-based educational tools are limited to low-dimensional, abstract tasks and lack the high-fidelity 3D physics engines required for realistic robot control, creating a disconnect between accessible web interfaces and the computational demands of robotics.

2. Methodology: Web-Gewu Architecture

The authors propose Web-Gewu, a browser-based interactive platform built on a Cloud-Edge-Client collaborative architecture utilizing WebRTC. The system fundamentally restructures the computational topology to separate heavy processing from user interaction.

A. Three-Tier Topology

1. Edge Tier (Compute Layer):
   • Hosted on a local GPU-equipped workstation.
   • Executes all resource-intensive tasks: Unity runtime, high-fidelity rigid-body physics simulation, RL training/inference logic, and hardware-accelerated video encoding (H.264).
   • Key Innovation: The cloud server performs no rendering or physics calculations.
2. Cloud Tier (Signaling Layer):
   • Acts exclusively as a lightweight signaling relay for NAT traversal.
   • Runs frps (for port mapping) and coturn (STUN/TURN services).
   • Efficiency: A standard 2-core vCPU instance with 2 GB RAM suffices, sustaining <1.1% CPU utilization even with active sessions.
3. Client Tier (Browser Layer):
   • A zero-installation web interface.
   • Receives real-time video streams (VideoTrack) and sends control commands (DataChannel) via Peer-to-Peer (P2P) WebRTC connections.
   • Handles UI interaction and telemetry visualization without local GPU requirements.

B. Low-Latency Streaming & Control

• Video Pipeline: Unity's Render Streaming plugin captures the simulation camera, encodes frames via hardware-accelerated H.264, and streams them directly to the browser's <video> element. This eliminates client-side scene reconstruction.
• Connection Strategy: Uses standard ICE procedures. Under favorable conditions, media and control traffic flow via direct P2P UDP (NAT hole-punching), ensuring end-to-end latency is imperceptible. TURN relays are used only as a fallback for strict firewall environments.

C. Robust Communication Protocol

• Envelope Protocol: A typed JSON protocol wraps all messages with metadata (version, ID, type, source, timestamp).
• Command Semantics:
  • State-Intent Commands (e.g., scene.load): Deduplicated by the server to prevent cascade failures from network jitter.
  • Continuous Control Snapshots: The receiver discards stale packets and applies only the most recent state, naturally tolerating packet loss without retransmission overhead.
• Synchronization: Browser commands are buffered and flushed atomically; Unity processes inbound callbacks sequentially on the main thread to ensure thread safety.

D. Layered Engineering Architecture

The system is organized into five decoupled layers to ensure extensibility:

1. Transport Layer: Manages WebRTC lifecycle and JSON serialization.
2. Protocol Layer: Defines message formats and dispatches by type.
3. Orchestration Layer (SceneDirector): Manages scene loading, transitions, and command replay.
4. Scene Runtime Layer: Executes specific physics/RL logic (e.g., specific robot agents).
5. UI Interaction Layer: Maps user gestures to payloads and renders telemetry.

• Extensibility: New scenarios can be onboarded by registering a scene, exposing a runtime object, and adding a single typed handler, without modifying core WebRTC or protocol code.

3. Key Contributions

1. Zero-Installation Platform: A cross-platform, browser-based environment supporting real-time control and gait verification for multi-form robots (bipedal, quadrupedal, leg-wheeled).
2. Cost-Efficient Architecture: A novel topology where the edge handles all heavy computation and the cloud acts only as a signaling relay, breaking the cost bottleneck of traditional high-fidelity cloud rendering.
3. Dynamic Visualization Loop: Real-time transmission of multi-dimensional monitoring data (reward curves, curriculum stages) via WebRTC DataChannels, making the data-driven nature of RL intuitive for learners.
4. Minimal Integration Contract: A framework allowing researchers to integrate new Unity-based robot scenarios with minimal engineering effort (register, bind, add handler), fostering community extensibility.

4. Results & Evaluation

• Infrastructure Overhead: Validation showed the cloud server CPU utilization remained stable at 0.8%–1.1% during active sessions, confirming the economic viability of the architecture compared to GPU-heavy cloud rendering.
• Educational Scenarios: The platform successfully demonstrated three distinct scenarios:
  • Playground: Real-time inference of pre-trained policies for various robots.
  • RoboHeTu: Navigation of a G1 quadruped in a complex virtual environment inspired by Along the River During the Qingming Festival, with real-time gait adaptation.
  • TinkerCoin: Live RL training of a bipedal robot using a Curriculum Learning strategy with a decaying assistive force.
    • Outcome: The robot achieved stable forward locomotion in ~3 minutes and fully unsupported walking in ~8 minutes.
    • Visualization: Real-time reward and episode length curves clearly visualized the transition from exploration to exploitation, showing the robot sustaining upright walking well before the curriculum support was fully withdrawn.
• Latency: The system achieved low end-to-end latency with no perceptible lag, enabling real-time intervention in robot behavior.

5. Significance

Web-Gewu resolves the long-standing tension between computational accessibility and high-fidelity simulation. By shifting the computational burden to the edge and utilizing WebRTC for P2P streaming, it democratizes access to cutting-edge robotics technology.

• For Education: It lowers the barrier to entry, allowing students to interact with complex RL algorithms and physical simulations on any device with a browser, without needing expensive hardware.
• For Research: It provides a scalable, open framework for deploying and visualizing RL training in real-time, bridging the gap between abstract algorithmic concepts and observable physical behaviors.
• Future Impact: The architecture lays the groundwork for future Sim2Real deployments and community-driven expansion of robot scenarios, positioning itself as a foundational infrastructure for embodied intelligence education.