Constrained Scenario Adaptation in Flight Simulator Training using Conceptualized State Spaces

Technical Summary: Constrained Scenario Adaptation in Flight Simulator Training using Conceptualized State Spaces

Problem Statement
The aviation sector, guided by the EASA AI Roadmap 2.0, requires machine learning (ML) systems to be not only accurate but also deterministic, interpretable, traceable to formally defined concepts, bounded by explicit safety constraints, and reconstructable post-hoc. Current approaches to adaptive pilot training often treat ML as an unconstrained predictor over latent feature spaces, applying constraints only as post-hoc filters. This creates a methodological gap: there is no formally defined transformation that maps heterogeneous training evidence (combining theory-based assessments and simulator telemetry) into admissible scenario adaptations while simultaneously ensuring determinism, semantic grounding, constraint compliance, and reconstructability. Existing conceptualization approaches for cyber-physical systems (CPS) do not explicitly address how to integrate such semantically grounded state spaces into safety-critical ML architectures.

Methodology
The paper proposes a formal mapping $\Phi: D_{raw} \rightarrow \Delta_{scenario}$ that decomposes the transition from raw data to scenario adaptation into four structurally verified substeps. The architecture is instantiated for stall recognition and recovery training within the Evidence-Based Training (EBT) and Competency-Based Training and Assessment (CBTA) paradigms.

1. Conceptualized State Space Construction (RQ1):
   The system transforms heterogeneous raw measurements ($D_{raw}$) into a deterministic, semantically grounded state space ($S$) using a three-step process:

   • Discretization: Continuous telemetry and assessment scores are converted into discrete diagnostic states ($Z$) via event detection and thresholding.
   • Semantic Assignment: Diagnostic states are mapped to a finite set of Deep-Linking Keywords ($K$) representing Knowledge, Skills, and Attitudes (KSA). This links theory-based knowledge work with simulator-based performance.
   • Aggregation: Evidence is aggregated into a unified learner state ($s_i$) containing KSA scores, maneuver-specific trace features, instructional progress, and context. This state serves as the input for the ML component, ensuring no latent or unconstrained representations are used.

2. Constrained Adaptation Model (RQ2):
   The adaptation logic is governed by a Deterministic Finite Automaton (DFA) that manages instructional phases independently of the ML layer. The adaptation policy operates within a strictly defined admissible action space ($A_{allowed}$) constrained by:

   • Phase Gating: Adaptation is only permitted in specific instructional phases.
   • Action Contracts: Scenario parameters (e.g., wind speed, altitude, maneuver repetition) are bounded by regulatory and safety limits.
   • Rule-Based Baseline: A deterministic rule-based policy ($\pi_{rule}$) serves as the reference, selecting focus keywords based on KSA profiles and applying threshold-based adaptation classes (strong, moderate, stable).

3. Constrained Policy Approximation (RQ2):
   An ML policy ($\pi_{ML}$) is trained to approximate the rule-based baseline ($\pi_{rule}$) within the constrained space ($\Pi_c$).

   • Learning Constraints: Monotonicity constraints are enforced during training (e.g., increasing skill must not increase the probability of remedial adaptation).
   • Inference Constraints: A deterministic projection operator ($P$) clips model outputs to the admissible action space and enforces phase gating. This ensures that even if the ML model predicts an inadmissible action, the executed policy ($\pi_{exec}$) remains compliant by construction.

4. Reconstructability (RQ3):
   Every decision is logged in a persistent trace ($\tau$) containing raw inputs, intermediate states, model outputs, and projection results, enabling full post-hoc auditability.

Key Contributions

1. Formal Mapping Definition: The paper defines the transition from heterogeneous training evidence to admissible scenario adaptation as a deterministic mapping $\Phi$ with four structural properties: determinism, semantic grounding, constraint compliance, and reconstructability.
2. Deterministic State Construction: It extends CPS conceptualization approaches by formalizing a three-stage state-space construction (discretization, semantic assignment, aggregation) that lifts heuristic keyword aggregation into a formal symbolic representation suitable for diagnosis and audit.
3. Architecturally Constrained ML: ML is embedded as a bounded approximation of a rule-based policy. Admissibility is guaranteed by construction through monotonicity constraints, deterministic instructional control, and a projection operator, rather than relying on post-hoc filtering.
4. Integrated Interpretability: Interpretability and auditability are treated as architectural properties derived from the state-space structure and persistent decision traces, rather than as post-hoc add-ons.

Results
The framework was empirically validated as a proof-of-concept using $N=160$ policy cases derived from stall training scenarios, employing group-wise $k=5$ cross-validation to prevent data leakage.

• Performance: The constrained ML policy approximated the deterministic baseline with a profile Mean Absolute Error (MAE) of $0.126 \pm 0.024$.
• Classification: The model achieved a focus-classification accuracy of $0.631 \pm 0.080$, representing a lift of $+0.344$ over a majority-class baseline ($0.288 \pm 0.042$).
• Semantic Analysis: SHAP analysis revealed that while certain keywords (e.g., "energy-management") were rarely selected as explicit focus targets, they acted as significant policy-shaping variables, indicating the model captures complex semantic interactions rather than simple rule replication.
• Verification: Analytical verification confirmed that the system satisfies the four structural properties (P1–P4) by construction, independent of the dataset size.

Significance and Claims
The paper claims to demonstrate that ML-based adaptation can be structurally compatible with safety-critical cyber-physical training systems without sacrificing determinism or auditability. The significance lies not in unrestricted adaptivity, but in defining formally specified conditions under which ML can operate within a safety-critical framework. The authors explicitly state that the empirical results are a proof-of-concept; the sample size ($N=160$) is sufficient to demonstrate operationalizability and constraint compliance but does not support claims of statistical generalization across different cohorts or maneuver categories. The work identifies multi-cohort generalization and the formal characterization of correctness conditions for adaptation decisions as necessary next steps. The proposed architecture offers a general methodological pattern for integrating constrained ML into safety-critical CPS, specifically addressing the EASA requirement for trustworthy, traceable, and bounded AI in aviation.