Technical Summary: Robust and Reliable AI for Predictive Quality in Semiconductor Materials Manufacturing

Problem Statement

Semiconductor materials manufacturing faces unique challenges in deploying machine learning (ML) models due to evolving process conditions, equipment degradation, and raw material variability. These factors lead to data and concept drift, causing model performance to deteriorate over time. A critical limitation of traditional ML approaches in this domain is the inability to quantify prediction uncertainty, which is essential for risk-aware decision-making in high-stakes production environments. Furthermore, while Machine Learning Operations (MLOps) offers a structured approach to model maintenance, there is a lack of comprehensive benchmarks for retraining strategies tailored to the specific characteristics of manufacturing data, such as seasonal variations, maintenance cycles, and supply chain fluctuations.

Methodology

The study utilizes five years of real-world manufacturing data from a high-volume production process of formulated semiconductor chemicals (approximately 1,200 batches). The dataset includes raw material (RM) quality control parameters, environmental factors, and final product (FG) quality targets.

Core Components:

1. Predictive Modeling: Random Forest (RF) regression was selected as the primary algorithm due to its robustness in handling feature interactions and its suitability for production environments requiring data efficiency. Hyperparameters were optimized using nested cross-validation with random search.
2. MLOps Benchmarking: The study systematically evaluates retraining strategies across three dimensions:
   • Retraining Cadence: Frequencies ranging from no retraining to every 5, 100, or daily batches.
   • Training Data Window: Comparison between fixed-size sliding windows (100 batches) and expanding training sets.
   • Hyperparameter Optimization: Evaluation of full model re-optimization versus fixed hyperparameter configurations during retraining.
3. Uncertainty Quantification: Conformal prediction (specifically the CV+ method via the MAPIE library) was implemented to generate prediction intervals with statistical guarantees. This approach does not assume specific residual distributions (e.g., Gaussianity).
4. Evaluation Metrics:
   • Point Prediction: Performance is measured using control limit normalized residuals (absolute residual divided by the Control Limit Range, CLR). A three-tier classification system (Good, Mediocre, Poor) is defined based on residual magnitude relative to the CLR.
   • Uncertainty: Metrics include coverage (proportion of actual values within the interval) and interval width relative to the CLR.

Key Results

1. Retraining Cadence and Strategy

• Frequency: High-frequency retraining (every 5 batches) significantly outperforms lower frequencies or no retraining. It reduces "Poor" predictions (residuals > CLR/2) by approximately 25–30% in sudden shift scenarios and 10–15% in gradual drift scenarios compared to the baseline.
• Window Size: In scenarios with sudden distribution shifts (e.g., supplier changes), fixed-size sliding windows (100 batches) perform comparably to expanding windows at high retraining frequencies. However, for gradual drift, expanding windows showed more stable performance, whereas fixed windows exhibited volatility. The study concludes that for sudden shifts, fixed windows are preferred to avoid computational overhead and potential degradation from obsolete data.
• Hyperparameter Tuning: Retuning hyperparameters during retraining provided negligible improvement for "Mediocre" predictions but offered a 10–20 percentage point improvement for "Poor" predictions. However, given the 40–60% increase in training time and the rarity of "Poor" cases in well-maintained systems, the authors conclude that hyperparameter retuning is inefficient for routine updates.

2. Uncertainty Quantification

• Out-of-Control (OOC) Detection: Integrating conformal prediction dramatically improved the detection of OOC batches. While traditional point prediction achieved a sensitivity of only 1.9% for unforeseen OOC events, the conformal approach achieved over 80% sensitivity (a >40-fold improvement).
• Coverage and Width: The conformal framework consistently maintained coverage rates above the theoretical 90% target across various training set sizes. However, conditional coverage analysis revealed heterogeneity: coverage dropped below the target level near the control limits (UCL/LSL), correlating with higher RMSE in those regions. This suggests that while the model is reliable on average, its ability to predict extreme values near boundaries is limited by the underlying regression model's accuracy and potential unobserved variables.

Key Contributions

• Empirical Benchmarking: The paper provides a systematic benchmark of MLOps retraining strategies (cadence, window size, hyperparameter tuning) specifically for semiconductor materials manufacturing, identifying that a fixed cadence of every 5 batches without hyperparameter retuning offers the optimal balance of accuracy and computational efficiency.
• Uncertainty Integration: It demonstrates the practical implementation of conformal prediction in a manufacturing setting, showing how prediction intervals can transform reactive quality control into a proactive framework by identifying batches where the prediction interval exceeds control limits.
• Operational Guidelines: The study offers practical guidelines for maintaining model reliability in dynamic environments, emphasizing that frequent, scheduled retraining is more effective than reactive, drift-triggered retraining in contexts where drift is subtle or masked by noise.

Significance and Claims

The authors claim that their findings provide practical guidelines for implementing robust MLOps strategies where computational efficiency and reliable uncertainty quantification are paramount. The study asserts that:

• Regular, high-frequency retraining is essential to prevent rapid model quality deterioration in the face of data drift.
• Conformal prediction is a versatile and reliable method for adding statistically rigorous confidence intervals to established regression models, significantly enhancing the ability to predict and prevent costly out-of-control events.
• While conditional coverage limitations exist near control limits, the overall improvement in sensitivity to OOC events makes uncertainty quantification a critical component of predictive quality systems.

The paper concludes that these methodological improvements ensure the resilience of ML-based predictive quality systems in real-world manufacturing scenarios, enabling proactive decision-making regarding raw material selection and blending ratios.