🔬 materials science

Probabilistic Deep Learning Framework for Phase Transformation Forecasting aided by In Situ High temperature Microscopy

This paper introduces a probabilistic deep learning framework that unifies image compression, temporal prediction, and microstructure forecasting to accurately predict the evolution of Widmanstätten ferrite in S235 steel under arbitrary thermal histories using in situ high-temperature microscopy data.

Original authors: Ioannis Kouroudis, Niki Balestrieri, Stefan Rotzsche, Niccolo Radice, Peter Mayr, Alessio Gagliardi

Published 2026-09-16

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Ioannis Kouroudis, Niki Balestrieri, Stefan Rotzsche, Niccolo Radice, Peter Mayr, Alessio Gagliardi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The strength, toughness, and durability of the steel in a bridge, a car frame, or a skyscraper are not determined solely by the chemicals mixed into the molten metal. They are decided by the invisible architecture that forms inside the metal as it cools. When steel is heated and then allowed to cool, its internal structure rearranges itself into different shapes and patterns, much like water freezing into ice crystals. The speed at which the metal cools dictates which patterns emerge. If it cools quickly, one type of crystal forms; if it cools slowly, a different one takes over. These microscopic patterns, known as phases, are the ultimate judges of how the material will behave under stress. For engineers, predicting exactly how these patterns will form is a high-stakes challenge. Traditional methods rely on static charts that assume a steady, unchanging cooling speed, but real-world manufacturing often involves complex, shifting temperatures that these charts cannot capture.

A team of researchers at the Technical University of Munich has developed a new way to solve this problem by teaching a computer to watch the steel cool in real time and predict its future. Instead of relying on static diagrams, they used a powerful microscope to film the surface of S235 steel as it transformed from a hot, molten state into a solid structure. They then fed these video recordings, along with the exact temperature history of the cooling process, into a sophisticated artificial intelligence system. This system learned to compress the complex visual details of the steel's surface into a simple summary, which it then used to forecast how the internal structure would evolve second by second. The result is a tool that does not just guess the final outcome but predicts the entire journey of the transformation, including how certain or uncertain that prediction is at any given moment.

The researchers focused on a specific type of steel structure called Widmanstätten ferrite, which appears as needle-like shapes growing within the metal. This structure is common in structural steels and significantly influences their strength. To understand how it forms, the team heated small samples of S235 steel to 1,200 degrees Celsius, a temperature high enough to melt the internal grain structure, and then let them cool down at five different speeds. They used a high-temperature confocal laser scanning microscope to capture the changes on the surface. This specialized microscope uses a laser to create a sharp image of the metal even while it is glowing hot, allowing the scientists to see the needle-like structures appear and grow without the interference of the metal's own heat radiation. They recorded these events at a rate of ten frames per second, creating a continuous movie of the transformation for each cooling speed.

To make sense of these movies, the researchers first had to teach a computer how to identify the needle-like structures automatically. They developed a method that scans each frame of the video, looking for specific changes in contrast that signal the presence of the ferrite. By counting the pixels that match this pattern, the system calculated exactly what percentage of the surface was covered by these structures at every moment. They confirmed this method was accurate by comparing the computer's counts with traditional laboratory photos of the same steel samples after they had cooled and been chemically treated to reveal their full structure. The data showed a clear pattern: the faster the steel cooled, the more of these needle-like structures formed. When the cooling was rapid, taking about 19 seconds to drop from 800 to 500 degrees Celsius, the steel ended up with about 51 percent of this structure. When the cooling was slow, taking nearly 100 seconds, the amount dropped to about 28 percent.

The core of the new framework is a two-part artificial intelligence system designed to predict this evolution. The first part takes the initial image of the steel's surface before it starts cooling and compresses it into a compact digital summary. This summary captures the unique starting condition of that specific piece of metal. This summary, along with the cooling speed, is then passed to a second, more complex system that acts as a time-traveling predictor. This system watches the temperature history as it unfolds and forecasts how the percentage of needle-like structures will change over time. It does not just give a single number; it provides a range of likely outcomes, showing the most probable path and the boundaries of uncertainty. To ensure the prediction is accurate at the very end, a third component checks the final state and corrects the forecast if the main system drifts slightly off course.

The results of this approach were remarkably precise. When the researchers tested the system on steel samples it had never seen before, the predicted paths of the transformation matched the actual observed paths with high fidelity. The system correctly identified when the transformation began, how fast it grew, and when it stopped. In a statistical test comparing the predicted values against the real measurements, the model achieved a score of 0.955, indicating a very strong agreement. The system also successfully quantified its own uncertainty. During the periods when the steel was stable and nothing was changing, the range of possible outcomes was very narrow. However, as the transformation began and the needle-like structures started to grow rapidly, the range of uncertainty widened, reflecting the natural variability of the process. This ability to show not just what will happen, but how confident the model is in that prediction, is a significant step forward.

This work demonstrates that it is possible to move beyond static charts and simple guesses to a dynamic, probabilistic understanding of how materials change. By combining high-speed imaging with advanced machine learning, the researchers created a framework that can track the life of a material's internal structure as it happens. While the current study focused on one type of steel and one specific structural pattern, the method is designed to be adaptable. The researchers suggest that this same approach could eventually be applied to other materials and more complex heating cycles, such as those found in 3D printing or welding. The ultimate goal is to provide engineers with a tool that can guide the manufacturing process in real time, allowing them to adjust the cooling speed on the fly to achieve the exact material properties they need. This represents a shift from simply observing the past to actively shaping the future of material performance.

Technical Summary: Probabilistic Deep Learning Framework for Phase Transformation Forecasting

Problem Statement
The mechanical performance of structural materials, particularly steels like S235, is governed by their microstructure, which evolves based on thermal history. While conventional Continuous Cooling Transformation (CCT) diagrams and thermodynamic models (e.g., Johnson-Mehl-Avrami-Kolmogorov) provide static descriptions of phase transformations, they suffer from significant limitations. CCT diagrams are typically measured for single compositions and constant cooling rates, failing to capture arbitrary, non-monotonic thermal cycles common in welding and additive manufacturing. Furthermore, they lack information on microstructural morphology, spatial distribution, and grain heterogeneity. Existing deep learning approaches often treat image-based characterization, temporal prediction, and microstructure forecasting as separate tasks, focusing on final states rather than the dynamic evolution of microstructures under arbitrary thermal trajectories. There is a methodological gap in unifying these tasks to forecast the full transformation trajectory with uncertainty quantification using in situ high-temperature microscopy data.

Methodology
The authors propose a probabilistic deep-learning framework that unifies image-based characterization and temporal prediction for in situ high-temperature confocal laser scanning microscopy (HT-CLSM) data. The pipeline consists of three main components:

Data Acquisition and Preprocessing:
- Experiments were conducted on S235 steel using HT-CLSM, capturing video at 10 fps during cooling from 1200°C under five distinct cooling regimes (Settings 1–5), characterized by $t_{8/5}$ times ranging from ~19s to ~99s.
- Initial surface images were processed using a Python/OpenCV pipeline. Frames were sampled at 1 fps, converted to grayscale, normalized via histogram equalization, and cropped to 800×800 pixels.
- A gradient masking routine identified Widmanstätten ferrite laths by thresholding local contrast gradients, computing the Widmanstätten ferrite ratio (area fraction) for each frame as the ground truth.
Model Architecture:
- $\beta$ -Variational Autoencoder ( $\beta$ -VAE): The initial surface image is compressed into a low-dimensional latent representation ( $d=20$ ). The $\beta$ -VAE (with $\beta=3$ ) imposes stronger compression toward a standard normal prior, ensuring a smooth, continuous latent manifold where similar images map to nearby vectors. This serves as a static covariate representing the initial microstructure.
- Temporal Fusion Transformer (TFT): The latent image representation, cooling rate, and the full temperature history $T(t)$ are fed into a TFT. The TFT utilizes LSTM encoders for local dependencies and multi-head attention for global dependencies. It outputs a quantile forecast (specifically 0.05, 0.25, 0.5, 0.75, 0.95) of the Widmanstätten ferrite ratio over time, capturing the dynamic evolution.
- Gaussian Process (GP): In parallel, the same latent representation and cooling rate are passed to a GP with an MLP kernel. The GP predicts the end-state (equilibrium) Widmanstätten ferrite ratio as a single scalar.
- Fusion Strategy: The TFT provides the dynamic evolution profile, while the GP corrects the equilibrium level. The model identifies the saturation point (where the gradient of the median TFT trajectory is near zero and time exceeds a guard period) and replaces the TFT prediction with the GP equilibrium estimate from that point onward. This yields a final sequence with corrected point estimates and probabilistic prediction intervals.

Key Contributions

Unified Framework: The work introduces a single pipeline that integrates image-based characterization, temporal prediction, and microstructure forecasting, moving beyond the separation of these tasks in prior literature.
Probabilistic Forecasting: Unlike deterministic CCT diagrams or single-point regression models, the framework provides trajectory-specific forecasts with uncertainty quantification (prediction intervals). It distinguishes between deterministic phases (plateaus) and stochastic phases (transformation ramps).
Hybrid Architecture: The combination of a $\beta$ -VAE for static image compression, a TFT for temporal dynamics, and a GP for equilibrium correction addresses the trade-off between recurrence and attention mechanisms while ensuring physical consistency with the final state.
In Situ Application: The method is specifically designed for and validated on temporally continuous HT-CLSM data, capturing spatial and temporal dimensions simultaneously under controlled thermal cycles.

Results
The framework was evaluated on S235 steel across five cooling settings (four repeats each) using a setting-level 5-fold cross-validation to ensure generalization to unseen thermal histories.

Microstructural Trends: Experimental analysis confirmed a monotonic increase in Widmanstätten ferrite fraction with increasing cooling rate (from ~28% at 3 K/s to ~51% at 16 K/s), consistent with metallurgical mechanisms where faster cooling suppresses diffusional growth and promotes side-plate formation.
Prediction Accuracy: The equilibrium-corrected TFT model achieved an overall coefficient of determination $R^2 = 0.955$ , a Mean Absolute Error (MAE) of 0.021, and a Root-Mean-Square Error (RMSE) of 0.038 across all test samples.
Temporal Dynamics: The model accurately tracked the transformation onset, sigmoidal growth, and saturation plateau. The probabilistic intervals were narrow in pre-transformation and plateau regions but widened during active transformation, reflecting kinetic variability. The 5–95% intervals consistently enveloped the ground-truth trajectories.
Limitations: Performance showed some scatter at extreme ratios (>0.45 and <0.1), attributed to the sparsity of data at extremes and the difficulty of localizing abrupt displacive transformations. The ground truth is limited to 2D surface sections.

Significance
The paper claims that this framework verifies the relevance of unifying image and temporal data for phase transformation forecasting. By providing probabilistic, trajectory-specific predictions rather than static summaries, the method offers a more informative tool for understanding transformation kinetics (onset, growth rate, saturation). The authors state that this approach paves the way for streamlined and highly accelerated investigation of material evolution under thermal treatment. It enables the prediction of responses to previously unseen thermal histories and opens the possibility of monitoring and adapting thermal cycles in real-time. The work serves as a proof of concept for data-driven decision support in materials processing, with future potential to extend to additional phase constituents and online thermal profile optimization.

More like this