⚡ electrical engineering

Sim-to-Real Transfer Learning for Real-Time Melt Pool Geometry Prediction in Directed Energy Deposition

This paper presents a Sim-to-Real transfer learning framework that leverages a multi-input hybrid CNN pretrained on physics-based simulations and fine-tuned on limited experimental data to achieve accurate, real-time prediction of melt pool width and depth in Directed Energy Deposition, thereby overcoming the long-standing barrier of indirect depth measurement for closed-loop control.

Original authors: Youmna Mahmoud, Nikhil Muralidhar, Chaitanya Krishna Vallabh, Souran Manoochehri

Published 2026-08-25

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Youmna Mahmoud, Nikhil Muralidhar, Chaitanya Krishna Vallabh, Souran Manoochehri

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Metal manufacturing has long relied on the ability to melt and fuse materials with extreme precision, a process that builds complex parts layer by layer. In one advanced technique known as directed energy deposition, a powerful laser beam melts metal powder as it is sprayed onto a surface, creating a small, glowing pool of molten metal. The shape and size of this molten pool determine whether the final part is strong and flawless or riddled with hidden cracks and weak spots. The problem is that the most critical part of this pool—the depth to which the metal melts—happens beneath the surface, invisible to the naked eye and impossible to measure while the machine is running. For decades, engineers have had to stop the machine, cut the part open, and examine it under a microscope to see if the process worked, a slow and destructive method that leaves no room for real-time correction.

A team of researchers at Stevens Institute of Technology has developed a new way to see the invisible. They created a system that uses artificial intelligence to predict the hidden depth of the molten pool in real time, using only the heat patterns visible on the surface and the settings of the laser. By teaching a computer to learn from a vast library of computer simulations first, and then refining that knowledge with a small amount of real-world data, they built a model that can watch the printing process and instantly tell operators how deep the metal is melting. This approach bridges the gap between theoretical physics and the messy reality of a factory floor, offering a path toward manufacturing parts that are monitored and corrected as they are made, rather than inspected after they are finished.

The core challenge in this field is that the molten pool is a fleeting, chaotic event. As the laser moves across the metal, it creates a pool that is constantly shifting, with heat radiating outward and the metal flowing in complex ways. While cameras can easily see the width of the pool on the surface, the depth remains a mystery until the part is destroyed. To solve this, the researchers turned to a method called transfer learning. Imagine a student who spends years studying physics textbooks and solving thousands of practice problems in a quiet classroom before ever stepping into a real laboratory. That student already understands the fundamental rules of the world, so when they finally face a real experiment, they need far less time to learn the specifics of that particular setup. The researchers applied this same logic to their computer model.

First, they generated a massive dataset using a physics-based computer simulation. They modeled the behavior of stainless steel as it was heated by a laser, creating over 1,300 different scenarios with varying laser powers, speeds, and spot sizes. In this virtual world, the computer knew the exact depth and width of every molten pool it created. They used this simulated data to train a complex neural network, a type of artificial intelligence designed to recognize patterns. The network learned to associate the heat patterns on the surface with the hidden depth below, building a robust understanding of the physics involved. However, the researchers knew that a model trained only on perfect computer simulations would struggle when faced with the noise, reflections, and imperfections of a real machine.

To bridge this gap, they took the pre-trained model and fine-tuned it using a small set of real experimental data. They performed single-track printing experiments on a real machine, capturing high-speed infrared images of the molten pool. To teach the computer what the real pools looked like, they had to cut the printed tracks open and measure the actual depth and width under a microscope. Because they could not measure every single frame of the video, they used a statistical method to assign these measured values to the images, creating a training set of about 550 labeled images. When they fed this real data into the model, the network adjusted its internal settings to account for the differences between the clean simulation and the noisy real world.

The results showed that this two-step approach was far superior to training a model from scratch using only the limited real-world data. When the researchers tested the models on new, unseen printing jobs, the system trained with the help of simulations made significantly fewer errors. For predicting the width of the pool, the simulation-assisted model was accurate within about 7 percent, while the model trained only on real data was off by nearly 30 percent. The difference was even more dramatic for the depth, which is the harder variable to predict. The simulation-assisted model predicted the depth with an error of about 22 percent, whereas the model trained without simulation help missed the mark by nearly 75 percent. In the most difficult cases, where the data was extremely scarce, the simulation-trained model maintained a high level of accuracy while the other model failed to learn the correct patterns at all.

The researchers also discovered that the system worked best when it combined two types of information: the thermal images of the pool and the specific settings of the laser. The images provided the visual shape of the heat, which helped determine the width, while the laser settings provided the energy context needed to guess the depth. When they tested the system in real time on six different printing jobs that used settings the computer had never seen before, it successfully tracked the changing geometry of the pool as the laser moved. It correctly identified when the pool was too shallow, a condition that often leads to weak parts, while the unassisted model consistently overestimated the depth and missed these defects.

This work demonstrates that it is possible to bring the power of physics simulations into the real world of manufacturing without needing thousands of expensive, time-consuming experiments. By using simulations to teach the AI the rules of the game and real data to teach it the nuances of the specific machine, the researchers created a tool that is both accurate and efficient. The system does not just predict numbers; it offers a way to monitor the health of the manufacturing process as it happens, potentially allowing machines to adjust themselves on the fly to prevent defects. While the technology is still being refined, the study proves that combining the clarity of computer models with the reality of physical experiments can solve problems that were once considered too difficult to tackle in real time.

Technical Summary: Sim-to-Real Transfer Learning for Real-Time Melt Pool Geometry Prediction in Directed Energy Deposition

Problem Statement
In Directed Energy Deposition (DED), the geometry of the melt pool (specifically its width and depth) is a critical determinant of part quality, microstructure, and mechanical performance. However, real-time monitoring of melt pool depth is a persistent challenge because it is a subsurface feature that cannot be measured directly by surface sensors. Traditional monitoring relies on thermography or optical imaging, which are susceptible to noise, spatter, and emissivity variations, and often fail to provide accurate depth quantification without destructive cross-sectional analysis. While machine learning offers a data-driven solution, training robust models typically requires large, high-quality experimental datasets that are expensive, time-consuming, and destructive to acquire. Conversely, physics-based simulations can generate vast amounts of labeled data but suffer from a "sim-to-real" gap due to modeling simplifications and unmodeled process variabilities, limiting their direct deployment.

Methodology
The authors propose a Sim-to-Real Transfer Learning (TL) framework to bridge the gap between scalable simulation and realistic experimental data. The workflow consists of three primary stages:

Simulation Pretraining (Source Task): A large dataset (~1,360 samples) was generated using 3D thermal finite element simulations in COMSOL Multiphysics. These simulations modeled single-track DED processes on SS316L substrates, varying laser power (200–800 W), scan speed (200–1200 mm/min), and spot size (0.5–1.0 mm). The output included surface temperature maps and corresponding ground-truth melt pool width and depth values derived from the simulated 3D geometry.
Experimental Fine-Tuning (Target Task): A limited experimental dataset (~550 labeled frames) was collected using an Optomec LENS MTS 500 system equipped with an Optris PI 05M infrared (IR) camera. Due to the mismatch between the high temporal resolution of IR frames and the sparse nature of destructive cross-sectional measurements (taken at 25%, 50%, and 75% of the track), a statistical labeling strategy was employed. Ground-truth labels for individual frames were assigned by sampling from normal distributions based on the mean and standard deviation of the three cross-sectional measurements per track. To align experimental data with simulations, a temperature threshold of 1800°C was applied to IR images to suppress background noise and spatter.
Neural Network Architecture: A multi-input hybrid Convolutional Neural Network (CNN) was developed. It features two parallel branches: an image branch processing 400x400 IR thermal maps to extract spatial thermal gradients, and a process parameter branch processing numerical inputs (laser power, scan speed, spot size). These features are concatenated and passed through fully connected layers to regress melt pool width and depth.

Key Contributions

Novel Framework: The paper introduces a specific Sim-to-Real TL pipeline for metal AM that shifts the prediction target from temperature fields (common in prior PINN studies) to direct melt pool geometry (width and depth).
Hybrid Input Strategy: The study demonstrates that combining real-time thermal imagery with process parameters yields superior accuracy compared to using either input modality alone.
Statistical Labeling: The authors address the data scarcity and labeling mismatch problem by implementing a Gaussian sampling strategy to assign geometry labels to high-frequency IR frames, enabling the use of large volumes of experimental video data despite sparse destructive measurements.
Real-Time Deployment: The framework is validated not just on static test sets but via live, frame-by-frame prediction during the fabrication of previously unseen single tracks.

Results

Performance in Low-Data Regimes: The fine-tuned TL model significantly outperformed a baseline model trained from scratch on experimental data alone, particularly when training data was scarce (50–100 samples). For melt pool depth prediction with 100 samples, the TL model achieved an $R^2 > 0.6$ , whereas the baseline remained below 0.2.
Error Metrics: On six previously unseen single-track builds, the TL model achieved a Mean Absolute Percentage Error (MAPE) of 7.41% for width and 22.45% for depth. In contrast, the baseline model yielded MAPEs of 29.35% (width) and 74.66% (depth).
Convergence and Stability: The TL model demonstrated faster convergence and greater training stability, avoiding the overfitting observed in the baseline when data was limited.
Uncertainty Quantification: Using Monte Carlo dropout, the study showed that the fine-tuned model produced tighter 95% confidence intervals and lower predictive uncertainty compared to the baseline, particularly for depth predictions.
Ablation Findings: The ablation study confirmed that while process parameters alone could predict depth reasonably well, they failed to capture width variations. Conversely, images alone struggled with depth. The combined input provided the most balanced and accurate predictions.

Significance and Claims
The authors claim that this work establishes a practical pathway toward sensor-driven, closed-loop control in metal additive manufacturing. By leveraging physics-based simulations to pretrain models and refining them with limited experimental data, the approach overcomes the data scarcity barrier that typically hinders deep learning applications in DED. The results suggest that simulation-informed models can reliably predict subsurface melt pool geometry in real-time, enabling defect detection (e.g., lack of fusion or keyholing) and adaptive process control without the prohibitive cost of extensive experimental data collection. The paper positions this method as a viable alternative to expensive in-situ subsurface imaging techniques like high-speed X-ray, offering a scalable solution for industrial monitoring.

More like this