Technical Summary: Adaptive Digital Twins for Predictive Decision-Making

Problem Statement

Modern engineering systems, particularly in civil infrastructure, face significant challenges in monitoring structural integrity and managing operations under uncertainty. These systems are subject to material deterioration, cyclic loading, and extreme events, often exacerbated by climate change. Traditional monitoring approaches often rely on static models or offline-trained predictors that fail to account for the dynamic and uncertain nature of evolving structural conditions. Consequently, digital twins (DTs) that overlook these dynamics suffer from reduced personalization, lower accuracy, and diminished value in decision-making.

While existing DT frameworks utilize probabilistic graphical models (PGMs) and dynamic Bayesian networks (DBNs) to link physical assets with virtual representations, they often assume stationary transition dynamics. This assumption implies that the probability of a system moving from one state to another is fixed and known a priori. However, in reality, transition dynamics can be non-stationary and depend on evolving environmental conditions or unmodeled factors. Relying on precomputed optimal policies based on fixed transition models can lead to suboptimal decisions or even system failure if the initial assumptions diverge from ground truth. The core problem addressed is how to enhance the value realization of digital twins by introducing adaptivity into the state transition models, enabling the system to learn and update its understanding of dynamics in real-time.

Methodology

The proposed framework integrates Online Bayesian Learning with Parametric Markov Decision Processes (MDPs) within a Dynamic Bayesian Network (DBN) architecture.

1. Probabilistic Graphical Model (PGM) Framework

The interaction between the physical asset and its digital twin is modeled using a DBN with decision nodes. The system is defined by:

• Physical State ($S_t$): The hidden ground truth of the structural health.
• Digital State ($D_t$): A categorical random variable representing the inferred health condition (damage location and magnitude).
• Observations ($O_t$): Sensor data (e.g., vibration recordings) assimilated via Deep Learning (DL) models to update the belief in $D_t$.
• Actions ($U_t$): Control decisions (e.g., maintenance, traffic restriction) that influence the transition of the digital state.
• Rewards ($R_t$): A function quantifying the utility of the asset-twin system based on health status and action costs.

The information flow is bi-directional: physical-to-digital via data assimilation (inference) and digital-to-physical via control actions (planning).

2. Online Learning of Transition Dynamics

The core innovation lies in treating the state transition probabilities not as fixed parameters, but as random variables endowed with conjugate prior distributions.

• Parametrization: The transition model is parameterized using a Dirichlet-Multinomial distribution (or Beta-Bernoulli for specific cases). For a given action $u$, the transition probabilities are modeled as $\theta_{d,u} \sim \text{Dirichlet}(\alpha_{d,u})$.
• Bayesian Updates: As new observational data and state transitions are observed, the hyperparameters ($\alpha$) of the Dirichlet prior are updated in closed form. This allows the digital twin to "learn" the transition dynamics online, adapting to non-stationary processes without requiring a complete retraining of the underlying model.
• State-Independence: To ensure computational tractability for large state spaces, the paper employs a state-independent transition model where transition probabilities depend on the action and the step size (e.g., 0, 1, or 2 steps of deterioration) rather than the specific starting state, though the framework supports state-dependent formulations.

3. Dynamic Control Policies via Reinforcement Learning

Instead of computing a single stationary policy offline, the framework employs Model-Based Reinforcement Learning (MBRL) to solve a finite-horizon MDP.

• Policy Recomputation: At each time step, the optimal control policy $\pi^*_t$ is recomputed based on the current (updated) transition model.
• Objective: The policy maximizes the expected cumulative reward over the remaining lifespan of the structure, balancing structural safety (health rewards) and operational costs (action rewards).
• Algorithm: The solution utilizes value iteration (backward induction) to solve the Bellman optimality equations, yielding a time-dependent optimal policy that accounts for the remaining planning horizon.

Key Contributions

1. Adaptive Transition Modeling: The paper introduces a method to treat state transition probabilities as random variables with conjugate priors, enabling hierarchical online learning of transition dynamics through effortless Bayesian updates. This relaxes the stationarity assumption common in current DT literature.
2. Dynamic Policy Optimization: The authors formulate a parametric MDP where control policies are dynamically updated using MBRL. This moves beyond precomputed policies based on fixed dynamics, allowing the DT to adapt its decision-making strategy as it learns more about the system's behavior.
3. Mathematical Generalization: The work extends previous frameworks (e.g., [74]) to a broader class of probability distributions (specifically Dirichlet-Multinomial), making the approach applicable to a wider range of real-life, large-scale systems with complex state spaces.
4. Integrated Framework: The paper provides a complete computational pipeline, from offline training of DL models for state estimation to online Bayesian updating of transition models and dynamic policy recomputation.

Results

The methodology was assessed through a case study involving the Hörnefors railway bridge, a 15.7m concrete bridge in Sweden. The study simulated a 60-time-step lifespan with various control actions: Do Nothing (DN), Restrict Operations (RO), Minor Repair (MR), and Perfect Repair (PR).

• Performance Improvement: In a cluster of 100 simulations, the adaptive DT (using MBRL with precision updates) achieved a 17% higher cumulative reward compared to a non-adaptive DT using a fixed transition model and standard Dynamic Programming (DP).
• Robustness: The adaptive framework demonstrated zero failures (structural collapse defined as damage $\ge$ 80%) across all simulations. In contrast, non-adaptive policies were prone to failure when the initial prior transition model was poorly aligned with the ground truth.
• Tracking Accuracy: The adaptive DT achieved a mean tracking accuracy of 93.6% (with a 4% standard deviation) relative to the ground truth physical state, comparable to the non-adaptive counterpart (93.15%), confirming that adaptivity improves decision-making without sacrificing state estimation accuracy.
• Convergence: The posterior distributions of the transition probabilities converged toward the unknown ground-truth dynamics, significantly reducing uncertainty over time. The system successfully self-corrected when initial priors were inaccurate.

Significance and Claims

The paper claims that adaptivity is essential for the effective realization of digital twins in civil engineering. By enabling the DT to learn transition dynamics online, the framework ensures the virtual representation remains synchronized with the physical asset, even as conditions change.

The authors state that this approach offers enhanced personalization, increased robustness, and improved cost-effectiveness. Specifically, the ability to update control policies based on refined transition dynamics allows for more informed planning, preventing structural failure and optimizing maintenance schedules. The work positions itself as a step toward autonomous DTs that can handle non-stationary environments, moving beyond simple automation to systems capable of responding effectively to changing conditions over time. The authors note that while the current study focuses on long-term deterioration, future work will address extreme events and partial observability, further expanding the applicability of this adaptive framework.