AM-DefectNet: Additive Manufacturing Defect Classification Using Machine Learning -- A comparative Study

Technical Summary: AM-DefectNet

Problem Statement
Additive Manufacturing (AM), particularly processes like Laser Powder Bed Fusion (LPBF) and Directed Energy Deposition, faces significant challenges in ensuring production quality due to the complex, multi-physics nature of the melt pool dynamics. Defects such as lack of fusion (LOF), keyhole porosity, and balling phenomena compromise the structural integrity of printed parts. While traditional monitoring relies on in-situ (e.g., acoustic sensors, thermography) and ex-situ (e.g., X-ray, CT) techniques, these methods can be costly, time-consuming, and require extensive calibration. Furthermore, the intricate relationships between process parameters (laser power, scanning speed, hatch spacing), material properties, and melt pool geometry make it difficult to establish reliable, physics-based predictive models. There is a critical need for data-driven approaches that can accurately classify defects based on experimental datasets to optimize process parameters and improve part quality.

Methodology
The authors introduce AM-DefectNet, a comprehensive benchmarking framework designed to evaluate the efficacy of various Machine Learning (ML) models in classifying AM defects. The methodology involves the following key steps:

• Data Collection and Curation: The study utilizes a dataset compiled from peer-reviewed literature, primarily referencing work by Akbari et al. The dataset comprises approximately 2,019 data points, split into 1,514 for training and 505 for testing. Input features include processing parameters (laser power, scanning speed, layer thickness, beam diameter, hatch spacing) and material properties (thermal conductivity, specific heat, melting point, chemical composition). The target output is a four-class classification: Desirable (no defect), Keyhole, Lack of Fusion (LOF), and Balling.
• Featurization and Preprocessing: Categorical variables (e.g., process type: SLM vs. EBM) are converted using one-hot encoding. Both categorical and numerical inputs undergo min-max scaling. The dataset is partitioned using an 11-fold cross-validation strategy to mitigate overfitting and ensure robust performance estimation.
• Model Selection: The study evaluates 15 distinct ML models, categorized into:
  • Linear Algorithms: Logistic Regression and Support Vector Machines (SVM) with a linear kernel.
  • Non-Linear Tree-Based Algorithms: Decision Trees, Random Forest, AdaBoost, Bagging, Voting, and Gradient Boosting variants (XGBoost, LGBM, CatBoost).
  • Neural Networks: Multi-Layer Perceptrons (MLP/DNN) and various SVM kernels (RBF, Polynomial, Sigmoid).
  • Instance-Based Learning: k-Nearest Neighbors (k-NN).
• Evaluation Metrics: Model performance is assessed using Accuracy, Precision, Recall, and F1-Score, calculated via macro, micro, and weighted averages to account for class distribution. Learning curves are also employed to analyze model convergence and data requirements.

Key Contributions

1. AM-DefectNet Framework: The paper establishes a standardized benchmark for comparing ML models specifically for melt pool characterization and defect classification in AM, a comparison not previously explored in this comprehensive manner.
2. Comprehensive Model Comparison: The study systematically evaluates 15 algorithms, ranging from traditional linear classifiers to advanced ensemble methods and deep learning, providing a clear hierarchy of performance for this specific domain.
3. Identification of Optimal Algorithms: The research identifies that non-linear, tree-based ensemble methods, specifically CatBoost, LGBM, and XGBoost, significantly outperform linear models and other classifiers in this context.
4. Learning Curve Analysis: The authors provide insights into model behavior relative to dataset size, highlighting which models have reached convergence and which may benefit from additional data.

Results
The benchmarking results demonstrate a clear performance hierarchy among the tested models:

• Top Performers: CatBoost achieved the highest accuracy of 92.47%, followed by LGBM at 91.08% and XGBoost at 90.89%. These models also exhibited superior precision, recall, and F1-scores across the defect classes.
• Deep Learning Performance: The Deep Neural Network (DNN/MLP) demonstrated competitive performance with an accuracy of 88.55%, though it required careful hyperparameter tuning (including batch normalization and dropout) to mitigate overfitting.
• Underperforming Models: Linear models, such as Logistic Regression (62.17% accuracy) and SVM with a linear kernel (70.29% accuracy), struggled to capture the complex, non-linear relationships inherent in AM processes. SVMs with non-linear kernels (RBF, Polynomial) showed improvement but did not match the performance of the top tree-based ensembles.
• Confusion Matrix Analysis: Visual analysis of confusion matrices confirmed that CatBoost and LGBM effectively minimized misclassifications between defect types compared to other algorithms.

Significance and Claims
The paper claims that the study highlights the effectiveness of Machine Learning, particularly non-linear tree-based algorithms like CatBoost, in addressing the challenges of defect detection in Additive Manufacturing. By establishing the AM-DefectNet benchmark, the authors provide a validated foundation for process optimization. The results suggest that data-driven approaches can serve as a cost-effective and efficient alternative to traditional experimental monitoring for predicting melt pool characteristics and identifying defects such as LOF, balling, and keyhole formation. The study concludes that these ML models lay the groundwork for improving repeatability and reducing defect rates in AM production, though it modestly notes that further data may still be required to fully converge certain models and that the complexity of AM physics remains a challenge for linear approaches.