Embodied AI with Foundation Models for Mobile Service Robots: A Systematic Review

This paper presents the first systematic review of integrating foundation models into mobile service robotics, analyzing how these technologies address core challenges in perception and control, enabling applications in domestic and healthcare settings while discussing ethical implications and outlining future directions for safe, scalable, and trustworthy deployment.

The Gist

Imagine you have a very smart, well-read robot butler named "Robo." In the past, Robo was like a highly trained dog: you could teach him specific tricks like "sit," "fetch the newspaper," or "open the door," but only if you programmed him exactly how to do it. If you asked him to "bring me that thing over there," he would freeze because he didn't know what "that thing" was or where "over there" was.

This paper is about giving Robo a massive upgrade. Instead of just following a list of rigid commands, we are now teaching him to use Foundation Models. Think of these models as giving Robo a super-brain that has read almost every book, watched almost every movie, and seen almost every photo on the internet.

Here is the breakdown of this "super-brain" upgrade in simple terms:

1. The Big Problem: The "Translation Gap"

Before this upgrade, there was a huge gap between what humans say and what robots do.

• The Old Way: If you said, "Clean up the mess," the robot needed a pre-programmed list of exactly which objects were messy and exactly where to put them. If you said, "Get me the thing I dropped," the robot was lost.
• The New Way (Foundation Models): Now, Robo understands context. If you say, "I'm hungry," he doesn't just stand there; he understands you might want food, knows where the kitchen is, and can figure out how to open the fridge. He translates your vague, human language into specific, physical actions.

2. The Four Big Hurdles (and how the Super-Brain fixes them)

The paper identifies four main challenges in making these robots work in real life (like your home or a hospital) and explains how the new AI helps.

Challenge A: Understanding Vague Instructions

• The Analogy: Imagine asking a tourist for directions. If you say, "Go past the big red building," a normal robot might get confused if there are two red buildings.
• The Fix: The new AI acts like a local guide. It uses "common sense." It knows that if you say "bring me that," you are probably pointing at something, or that "that" refers to the object you just looked at. It can handle messy, incomplete sentences and still figure out what you mean.

Challenge B: Seeing and Hearing Everything at Once

• The Analogy: Imagine trying to drive a car while wearing sunglasses, listening to loud music, and someone is shouting directions at you. That's what a robot faces in a busy house or hospital.
• The Fix: The new AI is like a multitasking conductor. It can look at a video, listen to your voice, and feel the texture of an object all at the same time. It combines these senses so that if your voice is muffled by noise, it can still understand you by reading your lips or seeing your hand gestures.

Challenge C: Knowing When It's Unsure

• The Analogy: A dangerous robot is one that is overconfident. Imagine a robot trying to walk through a crowded hallway. If it thinks it's 100% sure it won't hit anyone, but it's actually wrong, it might crash.
• The Fix: The new AI is like a cautious driver. It knows when it's guessing. If the lights are dim and it can't see clearly, it will say, "I'm not sure if that's a person or a coat rack, can you help me?" instead of blindly crashing into it. This keeps everyone safe.

Challenge D: Running on a Small Battery

• The Analogy: These super-brains are usually huge, like a mainframe computer that fills a whole room. But a robot needs to fit in a backpack and run on a small battery.
• The Fix: Scientists are now teaching the AI how to be efficient. It's like taking a massive encyclopedia and shrinking it down to a smartphone app that still knows everything it needs to, without draining the battery in five minutes.

3. Where Will We See These Robots?

The paper looks at three main places where this technology is changing things:

• At Home: Imagine a robot that can help you cook dinner. You say, "Make me a sandwich," and it figures out which bread is fresh, finds the peanut butter, and doesn't drop the knife. It can also tidy up the living room, knowing that "put the toys away" means putting blocks in the blue bin and dolls in the basket.
• In Hospitals: Imagine a robot nurse that can deliver medicine to different rooms. It knows how to navigate a busy hallway, stop for a doctor pushing a cart, and find the right patient's room even if the layout changed yesterday. It can also monitor patients, noticing if someone looks like they are in pain or falling.
• In Public Places (Malls/Airports): Imagine a robot guide at an airport. You ask, "Where is Gate B12?" and it doesn't just point; it walks you there, avoiding crowds and telling you about delays along the way.

4. The "But..." (Ethics and Safety)

The paper also warns us that with great power comes great responsibility.

• Privacy: If a robot is in your house listening and watching everything, who owns that data? We need to make sure it doesn't leak your secrets.
• Trust: If the robot makes a mistake, who is to blame? The person who built it, the person who owns it, or the AI itself?
• Jobs: Will these robots take away jobs? The paper suggests they should be helpers (like a co-pilot) rather than replacements, helping humans do their jobs better and safer.

The Bottom Line

This paper is a roadmap for the future. It says: "We have the brains (Foundation Models) to make robots that can actually live with us, help us, and understand us. But we still need to teach them to be safe, efficient, and ethical before we let them into our homes and hospitals."

It's the difference between a robot that is a tool (like a toaster) and a robot that is a partner (like a helpful friend). We are finally building the partner.

Technical Summary

Here is a detailed technical summary of the paper "Embodied AI with Foundation Models for Mobile Service Robots: A Systematic Review" by Lisondra, Benhabib, and Nejat.

1. Problem Statement

The integration of Foundation Models (FMs)—including Large Language Models (LLMs), Vision-Language Models (VLMs), Multimodal Large Language Models (MLLMs), and Vision-Language-Action Models (VLAs)—into mobile service robots represents a paradigm shift in Embodied AI. However, despite rapid advancements, significant gaps remain in deploying these models in real-world, human-centered environments (homes, hospitals, public spaces).

The paper identifies four core challenges preventing the reliable deployment of foundation model-enabled mobile service robots:

1. Translation of Natural Language to Executable Actions: Difficulty in grounding ambiguous, colloquial, or underspecified human instructions into precise physical navigation and manipulation tasks, especially in dynamic contexts.
2. Multimodal Perception: The challenge of fusing heterogeneous sensor data (vision, speech, LiDAR, tactile) in real-time while handling noise, occlusions, lighting variations, and temporal desynchronization in crowded human environments.
3. Uncertainty Estimation: The inability of current systems to accurately quantify aleatoric (sensor) and epistemic (model) uncertainty, leading to overconfident predictions and unsafe behaviors in safety-critical scenarios.
4. Computational Constraints: The mismatch between the massive computational requirements of foundation models and the limited onboard hardware (energy, memory, latency) of mobile robots, which often precludes reliance on cloud computing due to privacy and latency issues.

2. Methodology

The authors conducted a systematic review of the literature to analyze the integration of foundation models in mobile service robotics.

• Data Collection: They queried the OpenAlex scientific indexing platform, retrieving 7,506 relevant works published between 1968 and 2025.
• Filtering: Works related to autonomous driving and aerial robotics were excluded to focus strictly on mobile service robots.
• Categorization: The literature was grouped by recurring technical themes to quantify research effort distribution.
• Analysis Framework: The review is structured around:
  • Identifying the four core challenges (Section 2).
  • Analyzing how specific foundation models (e.g., GPT-4, CLIP, RT-2, Magma, SAM-2) address these challenges (Section 3).
  • Examining real-world applications in three domains: Domestic Assistance, Healthcare, and Service Automation (Section 4).
  • Discussing ethical, societal, and human-interaction implications (Section 5).
  • Proposing future research directions (Section 6).

3. Key Contributions

The paper makes several distinct contributions to the field of robotics and AI:

• First Systematic Review on Mobile Service Robotics: Unlike previous surveys focusing on general robotics or stationary arms, this is the first review specifically targeting the unique constraints and opportunities of mobile service robots integrating foundation models.
• Unified Architecture View: The authors propose a unified architecture (Figure 2) illustrating how foundation models fuse multimodal inputs to drive perception, planning, and control simultaneously.
• Quantitative Challenge Ranking: Through publication analysis, the paper ranks the four core challenges by research volume:
  1. Language-to-Action Mapping (29.18%)
  2. Multimodal Perception (28.83%)
  3. Uncertainty Estimation (26.71%)
  4. Computational Capabilities (15.28%)
• Comprehensive Model Benchmarking: Table 2 provides a comparative analysis of major foundation models (e.g., Magma, DeepSeek-R1, SAM-2, CLIP-CAP, Perceiver-Actor) across the four challenges, highlighting trade-offs in success rates, latency, calibration error, and computational cost (GFLOPs).
• Application Taxonomy: The paper maps specific foundation model capabilities to sub-tasks in three critical domains:
  • Domestic: Fetch-and-carry, cleaning, childcare, cooking.
  • Healthcare: Medical delivery, patient monitoring, assistive mobility, infection control.
  • Service Automation: Wayfinding, customer assistance, venue setup.
• Ethical and Governance Framework: It outlines the ethical implications (privacy, accountability), societal impacts (labor displacement, equity), and human-interaction risks (emotional impact, manipulation), proposing a "Human-in-the-Loop" governance structure.

4. Key Results and Findings

Addressing the Challenges:

• Language-to-Action: Foundation models bridge the symbolic-to-embodied gap by learning shared semantic spaces. Models like CLIP-CAP (71% success rate in manipulation) and LLM-Planner enable robots to decompose high-level commands into modular control routines, handling ambiguity better than classical planners (STRIPS/PDDL).
• Multimodal Perception: Models like Magma and SAM-2 utilize unified latent spaces and attention mechanisms to align vision, language, and spatial data. This reduces representational misalignment and improves robustness against occlusions and domain shifts (e.g., from training labs to cluttered homes).
• Uncertainty Estimation: Recent models like DeepSeek-R1 demonstrate improved calibration (lower calibration error ~81.8% vs. ~92% in older models), allowing robots to express uncertainty and request clarification rather than acting overconfidently.
• Computational Efficiency: Techniques like Mixture-of-Experts (MoE) (Switch Transformer), model distillation (DeiT), and edge-optimized architectures (Perceiver-Actor) allow for onboard execution. Perceiver-Actor achieves multi-task control with only ~9-10 GFLOPs, making it viable for embedded systems.

Application Insights:

• Domestic: Robots can now infer user preferences (e.g., TidyBot) and perform long-horizon tasks (e.g., cooking with PaLM/UR5e) by grounding language in 3D semantic fields.
• Healthcare: Foundation models enable socially compliant navigation in crowded hospitals (VLM-Social-Nav) and hierarchical planning for disinfection (SayPlan), adapting to dynamic patient needs.
• Service Automation: Robots can interpret vague instructions for venue setup (Hi-Robot) and navigate complex public spaces using open-vocabulary scene graphs.

Limitations Identified:

• Hallucinations: LLMs can generate plausible but incorrect plans, posing safety risks in medical or navigation tasks.
• Catastrophic Forgetting: Online fine-tuning for new environments risks erasing previously learned skills.
• Data Scarcity: Sensitive domains (homes/hospitals) lack the large-scale labeled datasets required for training, necessitating synthetic data or federated learning.
• No "One-Size-Fits-All": No single model excels in all four challenges; trade-offs exist between accuracy, speed, and computational cost.

5. Significance and Future Directions

This review establishes a critical roadmap for transitioning foundation model-enabled robots from experimental prototypes to reliable, scalable assistants.

Future Research Directions Proposed:

1. Reliability and Lifelong Adaptation: Developing hallucination detection pipelines and continual learning methods (e.g., elastic weight consolidation) to prevent catastrophic forgetting in evolving environments.
2. Privacy-Aware and Resource-Constrained Deployment: Advancing federated learning, synthetic data generation, and task-aware scaling laws to operate efficiently on edge devices without compromising privacy.
3. Governance and Human-in-the-Loop Frameworks: Creating standardized benchmarks that include realistic human agents and unpredictable activities, alongside protocols for human oversight and override in safety-critical decisions.

Conclusion:
The paper concludes that while foundation models have successfully bridged the gap between high-level reasoning and low-level control, achieving safe, trustworthy, and scalable deployment requires addressing reliability, privacy, and governance. The integration of these models promises to transform mobile service robots into proactive, context-aware partners in daily life, provided that technical limitations are met with robust ethical and regulatory frameworks.