Does learning when to look pay for home service robots? A preregistered test of adaptive perception scheduling in closed-loop navigation

Technical Summary: Does Learning When to Look Pay for Home Service Robots?

Problem Statement
Home service robots operating on battery-powered embedded computers face a significant computational bottleneck: object detection is often the most expensive task in the perception pipeline. While adaptive perception strategies (skipping frames, cropping regions, or switching model sizes) are popular for reducing computational load, they are rarely evaluated in a closed-loop navigation setting. In static video analytics, a missed detection merely lowers recall; in a mobile robot, a missed detection can lead to task failure, stalling, or collision. Existing literature lacks a rigorous, preregistered evaluation of whether the computational savings of adaptive perception survive the constraints of closed-loop navigation, where the robot's motion depends entirely on its perception output.

Methodology
The authors conducted a preregistered experiment to test an adaptive perception controller against twelve baselines in a closed-loop navigation task.

• Environment and Setup: The study utilized the Habitat simulator with 25 photorealistic home scans (HM3D) held out from all training data. Obstacles (household objects) were placed outside the navigation mesh, meaning the local planner could only avoid them if the detector successfully identified them. The robot used an RGB-D camera (320×240 resolution) and a dynamic-window planner on a 0.1s control period.
• The Evaluated Controller: The proposed method learns when to run a detector, where in the image to look, and which model size to use.
  • WHEN: A learned trigger uses eight cheap signals (e.g., frame difference, ego-motion, time-to-collision) to predict if the current belief is missing a relevant obstacle.
  • WHERE: A hand-designed tile scorer selects specific image regions based on motion energy and braking corridors.
  • WHAT: A tier rule selects detector resolution based on time-to-collision (Heavy, Medium, or Light).
  • Safety Mechanisms: A "safety shield" forces a detection if the robot travels too far without looking, and an "experience memory" attempts to recall past scene outcomes.
• Baselines: The controller was compared against 12 baselines, including running the heavy or light detector on every frame, fixed-rate frame skipping, random frame selection, and various heuristic triggers (pixel difference, edge features, motion gating).
• Preregistration and Criteria: The task, validity gates, and success criteria were fixed in a preregistration before running the main comparison. To be considered successful, the adaptive method had to:
  1. Achieve a success rate within 0.05 of the best baseline (Always-Heavy).
  2. Achieve a collision rate within 0.05 of the best baseline (Always-Light).
  3. Use significantly fewer GFLOPs than the baselines.

Key Results
The study found that the adaptive controller did not meet the preregistered success criteria.

• Performance vs. Baselines: The full adaptive controller ("ours-full") succeeded in 55.0% of episodes and collided in 8.3%. In contrast, the "always-light" baseline (running the cheapest detector on every frame) succeeded in 76.7% of episodes with a collision rate of only 0.8%. The "always-heavy" baseline achieved a 79.2% success rate.
• Computational Savings: The adaptive controller reduced computation by 85% relative to the heavy detector (11,609 GFLOPs vs. 76,805 GFLOPs) and was significantly cheaper than 11 of the 12 baselines.
• Failure Modes: The adaptive method failed primarily by stalling behind false or stale tracks (unfinished tasks) rather than just colliding. The learned trigger did not converge effectively, and simple fixed-rate skipping often outperformed the learned trigger at equal computational budgets.
• Memory Transfer: The experience memory failed to generalize effectively to new homes; in fact, detector calls increased over repeated visits to the same scene, suggesting that compute needs are harder to predict from visual appearance than from robot position.

Significance and Claims
The paper argues that for home service robots, the "simple and cheap default" of running the cheapest adequate detector on every frame is currently the strongest baseline.

• Hardware Implications: On the measured processor (Apple M1 Pro), only the light detector tier completes within the 0.1s control period. The heavy tier's latency (244ms) exceeds the control period by 2.4x. Therefore, a robot designed around the light tier requires a smaller onboard computer and does not need a complex scheduler to stay within its budget.
• Design Guidance: The authors provide four practical guidelines:
  1. Start with the cheapest adequate detector running on every frame; adaptive schedulers must beat this default in closed-loop tests before deployment.
  2. Measure task completion (success rate), not just collisions, as poor perception often leads to unfinished tasks.
  3. Evaluate schedulers across their entire budget range, as simple fixed-rate skipping can outperform learned triggers at equal costs.
  4. Claims of cross-home experience transfer should be benchmarked against a position-keyed control.

Conclusion
The study concludes that learning when to look does not currently pay off for home service robots in terms of balancing safety, task success, and computation. The negative result is presented as a useful benchmark, establishing that any future adaptive perception method must first surpass the performance of a simple, low-cost detector running continuously in a closed-loop environment.