IDDMBSE: Integrating Data-Driven and Model-Based Systems Engineering for Trusted Autonomous Cyber-Physical Systems

Technical Summary: IDDMBSE – Integrating Data-Driven and Model-Based Systems Engineering

Problem Statement
Autonomous Cyber-Physical Systems (CPS) increasingly rely on hybrid architectures that interleave traditional model-based components (kinematics, planners, controllers) with data-driven Machine Learning and Artificial Intelligence (ML/AI) components (perception networks, learned policies). While Model-Based Systems Engineering (MBSE) provides a rigorous framework for complex system design, it lacks native methodologies to accommodate data-driven components. Conversely, ML/AI offers powerful inference capabilities but often operates as a "black box" within the system, lacking formal traceability to requirements. Current industrial practices typically treat ML/AI subsystems as black boxes retrofitted into model-based scaffolds, resulting in brittle systems, slow design cycles, and weak certification arguments. There is a critical gap in Systems Engineering (SE) methodology that natively spans both the rigorous, traceable V-process of MBSE and the empirical, data-driven nature of modern autonomy stacks.

Methodology: IDDMBSE
The paper proposes IDDMBSE (Integrated Data-Driven and Model-Based Systems Engineering), a methodology that extends the standard MBSE V-process by layering a data-driven loop at every step. The approach is anchored in SysML as the central representation and treats the autonomy stack as a first-class architectural element rather than an opaque payload.

The methodology is driven by a requirements repository, ensuring that data-driven efforts are deployed only where first-principles models are insufficient to guarantee requirement satisfaction. The framework integrates three structural extensions:

1. Data-Driven Augmentation: Every V-process step (structure, behavior, mapping, requirements, trade-off, verification) is augmented by ML/AI components, their training data, and uncertainty profiles.
2. Autonomy Stack Modeling: The perception-planning-control stack is modeled as a SysML block with structural, behavioral, and parametric ports that bind directly to executable implementations (e.g., ROS 2).
3. Hybrid Trade-off Architecture: Design space exploration combines model-based optimization (for tractable components) with data-driven evaluation (for learned components).

Key Contributions: The Tool Chain
The methodology is instantiated as an interoperable, open-source tool chain comprising three primary tools, all coupled through a shared SysML modeling hub:

1. PERFECT (PERFormance Evaluation Composable Tool-suite):

   • Function: Maps SysML system architectures to executable ROS 2 autonomy stacks for scalable performance evaluation.
   • Mechanism: It exposes a SysML profile that binds system blocks to ROS 2 nodes and parameters. PERFECT synthesizes launch configurations, distributes simulation jobs across a compute pool (using Isaac Sim or Gazebo), and aggregates results back into SysML parametric diagrams.
   • Innovation: Unlike standard benchmark suites, PERFECT maintains a "digital thread" where experiment configurations are derived from and feed back into the SysML model, ensuring traceability between architecture and measurement.

2. TRADES-X (TRadeoff Analysis and DEsign Space EXploration):

   • Function: Performs hybrid model-based and data-driven design-space exploration.
   • Mechanism: It decomposes the design space into two stages:
     • Model-Based Optimization (MBO): Uses first-principles models to prune the design space and generate a Pareto-optimal frontier of candidates.
     • Data-Driven Optimization (DDO): Runs statistical experiments on the surviving candidates using PERFECT as the execution backbone.
   • Innovation: It treats requirements as partitioned classes (model-based vs. data-driven) and uses automatic differentiation to rank requirements by sensitivity, grounding qualitative priorities in quantitative analysis.

3. VERITAS (VERIfication of Trusted Autonomous Systems):

   • Function: Provides a unified assurance workflow combining formal, data-driven, and runtime verification.
   • Mechanism:
     • Mathematical Verification: Compiles SysML diagrams into timed automata (UPPAAL) for formal proof.
     • Data-Driven Verification: Uses PERFECT simulation campaigns to estimate failure rates and confidence bounds for intractable formal models.
     • Runtime Monitoring: Synthesizes observers (e.g., Signal Temporal Logic monitors) to detect specification violations in the deployed stack.
   • Innovation: A design candidate is only considered "assured" if it survives all three verification modules, providing a multi-faceted assurance argument from a single source.

Results and Demonstrations
The framework was demonstrated on a Trusted Autonomous Ground Robot (AGR) across its development lifecycle, utilizing a contested-terrain test range in NVIDIA Isaac Sim:

• Sensor-Suite Selection (TRADES-X): The tool chain reduced 8,192 candidate sensor configurations to 92 Pareto-optimal candidates via MBO, and further to 7 survivors for data-driven evaluation, ultimately selecting a single recommended design.
• Risk-Sensitive Path Planning (PERFECT): The framework evaluated the RA-RRT* planner against RRT* across varying noise and risk levels. Results showed the risk-averse policy achieved lower worst-case path lengths and fewer failures, with PERFECT enabling the distributed campaign required for these comparisons.
• Behavior Tree Verification (VERITAS): The tool supported automated model generation and online monitor synthesis from Behavior Tree specifications, enabling formal verification of task plans.
• Robust Perception (Conformal Prediction): The system implemented conformal prediction to wrap object detectors with distribution-free guarantees. PERFECT generated the necessary closed-loop, ground-truth labeled data to calibrate the system, while TRADES-X optimized the trade-off between safety margins and navigation conservativeness.
• Assured Multi-Robot Coordination: The framework scaled to a team of robots, decomposing high-level fleet requirements into Signal Temporal Logic (STL) specifications. VERITAS verified executed trajectories, successfully identifying a slight violation in one robot's path that synthesis alone would have missed.

Significance and Claims
The paper claims that IDDMBSE addresses the root cause of the integration gap in autonomous CPS: the lack of a methodology that treats ML/AI components as first-class, requirements-driven citizens of the V-process. By integrating these components into a unified SysML representation, the framework enables:

• Traceability: Linking ML/AI components, system structure, and requirements through a single model.
• Scalability: Using model-based priors to prune the design space before expending scarce empirical resources on data-driven evaluation.
• Assurance: Closing the loop between formal models, statistical evidence, and runtime monitoring to support "Trusted Autonomy" (systems that self-monitor and self-adjust under verifiable safety guarantees).

The authors note that the current implementation relies on SysML v1 tooling, which introduces "brittle, GUI-bound" plumbing. Future work focuses on reformulating IDDMBSE on SysML v2 / KerML foundations to enable language-native composability, API-first integration, and tighter coupling with agentic AI for the V-process.