Technical Summary: Foundation Models in Robotics: A Comprehensive Review

Problem Statement

The field of robotics is undergoing a paradigm shift from fixed, single-task, domain-specific solutions toward adaptive, multi-function, general-purpose agents capable of operating in complex, open-world, dynamic environments. While traditional model-based control offers efficiency in structured settings, it lacks adaptability. Conversely, standard machine learning (ML) approaches offer high adaptability but often require extensive datasets and lack the semantic understanding necessary for open-world reasoning. The emergence of Foundation Models (FMs)—large-scale neural networks trained on massive, internet-scale, heterogeneous datasets—promises to bridge this gap by providing unprecedented capabilities in multi-modal understanding, long-horizon planning, and cross-embodiment generalization. However, the rapid proliferation of FM applications in robotics has resulted in a fragmented research landscape. Existing surveys often focus narrowly on specific tasks, models, or application domains, lacking a systematic, multi-criteria taxonomy that covers the full spectrum of methods, datasets, and challenges.

Methodology

This study employs a structured and systematic literature review methodology to map the research landscape of FMs in robotics. The process involved:

1. Literature Search: Queries were conducted across six major scientific databases (IEEE Xplore, Google Scholar, Scopus, DBLP, arXiv, and Web of Science) using targeted keywords (e.g., "foundation model," "robotics," "vision-language-action") combined with Boolean operators. The search prioritized works from the last five years while including seminal earlier studies.
2. Screening and Selection: A multi-stage screening process excluded duplicates, non-English papers, and articles without full-text access. Final selection retained 438 articles where an FM served as a key algorithmic component with substantial theoretical or experimental contributions.
3. Taxonomic Analysis: The selected literature was analyzed through a highly-granular, multi-criteria taxonomy. The review examined methods based on six distinct criteria:
   • Foundation Model Type (LLMs, VFMs, VLMs, VLAs).
   • Underlying Neural Network Architecture (Transformers, SSMs, Diffusion, CNNs, Graphical models).
   • Learning Paradigm (Supervised, Self-Supervised, Fine-tuning, Domain Adaptation, Imitation Learning, Reinforcement Learning, In-Context/Prompt Learning, World Models, Generative Learning).
   • Learning Stage (Pre-training, Offline Fine-tuning, Online Adaptation, Continuous Learning).
   • Robotic Task (Perception, Planning, Navigation, Manipulation, Human-Robot Interaction).
   • Application Domain (Agentic mobility, Industrial manipulation, Supply operations, Service robots, Medical robots, Cognitive agrisystems, Crisis agents, Maritime robotics, Space robotics).
4. Dataset Synthesis: Publicly available datasets were synthesized and organized by recurring families, detailing their typical uses and current gaps.
5. Challenge and Direction Analysis: A hierarchical discussion was formulated regarding current open challenges and promising future research directions.

Note: The authors explicitly state that this survey does not include empirical/quantitative benchmarking of the surveyed methods, acknowledging it as a snapshot of a rapidly evolving field.

Key Contributions

The paper provides a holistic and systematic investigation of the robotic FM landscape, offering the following specific contributions:

1. Evolutionary Framework: The research delineates the evolution of robotic FM research into five distinct phases:

   • Phase 1 (2018-2021): Integration of native NLP and CV models for perception support.
   • Phase 2 (2021-2022): Grounded planning using Vision-Language (VL) representations.
   • Phase 3 (2022-2023): Emergence of Embodied Vision-Language-Action (VLA) policies trained on large-scale robot data.
   • Phase 4 (2023-2024): Systems capable of memory, autonomous task composition, and Web-to-robot transfer.
   • Phase 5 (2024-Present): Multi-sensory generalization and real-world deployment.

2. Multi-Criteria Taxonomy: A comprehensive classification of methods is provided across six criteria. Key findings include:

   • Model Types: LLMs excel in high-level reasoning but lack physical grounding; VFMs offer robust perception but are domain-specific; VLMs bridge language and vision but require external policies; VLAs provide end-to-end policies but face high data and latency costs.
   • Architectures: Transformers dominate for multi-modal alignment; State-Space Models (SSMs) offer linear-complexity for real-time control; Diffusion models enable precise action generation; Graphical models support structured reasoning.
   • Learning Paradigms: A synthesis of Supervised, Self-Supervised, Imitation, Reinforcement, and Generative learning techniques is presented, highlighting the trade-offs between sample efficiency, generalization, and safety.

3. Dataset Landscape: The paper synthesizes public datasets, identifying a shift toward massive cross-embodiment trajectory corpora (e.g., Open X-Embodiment, AgiBot World) and high-fidelity real-world benchmarks. It critically notes current gaps, specifically the scarcity of tactile/force sensing data and the lack of failure/recovery recordings in existing datasets.

4. Application Domains: The review details the deployment of FMs across diverse sectors, including industrial manipulation (zero-shot generalization to novel objects), service robots (natural language instruction following), medical robotics (surgical depth estimation), and space robotics (autonomous terrain analysis).

5. Challenges and Future Directions: The paper identifies critical challenges, including inference latency, lack of semantic/physical grounding, data scarcity, embodiment bias, safety risks, and ethical alignment imperatives. Future directions are proposed toward multi-sensory integration, sim-to-real transfer, and robust real-world deployment.

Results and Insights

The review reveals that while FMs have fundamentally reformed robot design and programming, they are not a monolithic solution.

• Trade-offs: There is a fundamental trade-off between semantic breadth (provided by LLMs and VLMs) and real-time, embodied execution (provided by VFMs and VLAs). Hybrid systems are increasingly necessary to balance these attributes.
• Generalization vs. Precision: FMs enable zero-shot generalization to unseen objects and environments, often reducing the need for task-specific programming. However, this comes at the cost of reduced action precision in specialized domains and increased susceptibility to hallucinations and logical gaps in planning.
• Data Dependency: Despite the promise of "foundation" models, the field remains heavily dependent on the availability of large-scale, high-quality, multi-modal datasets. The scarcity of real-world failure data and tactile feedback remains a significant bottleneck for robust deployment.
• Evolutionary Trend: The field is moving from modular pipelines (separate perception and control) toward end-to-end, generalist policies that integrate perception, reasoning, and action within a single architecture.

Significance

The paper claims significance as the first survey to offer a holistic, systematic, and highly-granular multi-criteria investigation of the entire robotic FM landscape. Unlike prior works that focus on specific subsets (e.g., only VLAs or only manipulation), this study:

• Documents the full evolutionary trajectory of the field, including recent developments up to 2026.
• Provides a structured taxonomy that allows researchers to compare approaches across model types, architectures, learning paradigms, and application domains simultaneously.
• Synthesizes the current state of public datasets, highlighting critical gaps that hinder progress.
• Offers a comprehensive discussion on challenges and future directions, serving as a roadmap for researchers aiming to transition FMs from research prototypes to reliable, real-world robotic agents.

The authors emphasize that while the survey does not provide new empirical benchmarks, it establishes a necessary foundational understanding of the field's current state, limitations, and potential, facilitating more informed research and development in the era of foundation models for robotics.