🤖 machine learning

Comparative Evaluation of Machine Learning and Deep Learning Models for Wound-Rotor Synchronous Motor Performance Prediction

This study presents a comprehensive benchmark of eight machine learning and deep learning models for predicting wound-rotor synchronous motor performance, demonstrating that neural-network-based architectures, particularly the FT-Transformer, significantly outperform tree-based models in accuracy and computational speed while offering a robust multi-seed reproducibility protocol and Pareto analysis of cost-accuracy trade-offs.

Original authors: Kıvanç Doğan, Ahmet Orhan

Published 2026-06-23✓ Author reviewed ⓘ

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Kıvanç Doğan, Ahmet Orhan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to design the perfect engine for a new electric car. In the past, engineers relied on rare, expensive metals (like those found in magnets) to make these motors powerful. But because those metals are hard to get and can cause political trouble, engineers are now switching to a different type of motor called a Wound-Rotor Synchronous Motor (WRSM). Think of this new motor as a "smart" engine that doesn't need those rare metals but is much harder to design.

The Problem: The "Slow Math" Bottleneck

Designing this new motor is like trying to find the perfect recipe by tasting every possible combination of ingredients. There are dozens of variables (like the size of the metal plates, the number of wire turns, and the electric current).

Traditionally, engineers use a super-accurate but incredibly slow computer simulation called FEA (Finite Element Analysis) to test each recipe.

The Analogy: Imagine FEA is like a master chef who tastes a soup, analyzes every molecule, and gives you a perfect rating. The problem is, it takes hours to taste just one spoonful. If you need to test 3,000 recipes, you'd be waiting years.

The Solution: The "Fast Learners" (Machine Learning)

To speed this up, the researchers in this paper tried to teach computers to "guess" the results of the slow chef's tests. They used Machine Learning (ML) and Deep Learning (DL) models.

The Analogy: Instead of waiting for the master chef, they trained a group of "fast-tasting apprentices." These apprentices read thousands of past recipes and their results, learning patterns so they can predict the taste of a new recipe in milliseconds.

The Experiment: The Great Taste-Off

The researchers created a dataset of 3,351 valid motor designs (like a massive cookbook of recipes). They then pitted 8 different "apprentices" against each other to see who could predict the motor's performance (how much twisting force it makes and how efficient it is) most accurately.

The 8 apprentices came from four different "families":

Tree-Based Models (The Logic Puzzlers): These include models like CatBoost, LightGBM, and Random Forest. They make decisions by asking a series of "Yes/No" questions, like a flowchart.
Neural Networks (The Pattern Recognizers): These include MLP, PI-MLP, and the star player, FT-Transformer. These are designed to mimic the human brain, looking for complex, smooth connections between variables rather than simple yes/no steps.
Probabilistic Models (The Gamblers): Like Gaussian Process Regression, which tries to guess with a sense of uncertainty.

The Results: Who Won?

After running the tests 10 times to ensure the results weren't just luck, here is what happened:

The Champion: The FT-Transformer (a deep learning model) took the gold medal. It was the most accurate, predicting the motor's performance with near-perfect precision (99.28% accuracy).
- Why it won: The paper suggests that motor physics are like a smooth, flowing river. Neural networks are great at following that smooth flow. The "Tree" models, which chop the river into straight, jagged steps, struggled to capture the nuance.
The Runner-Up: Standard Neural Networks (MLP) came in second, still beating all the tree-based models.
The "Good Enough" Option: Among the tree-based models, CatBoost was the best. It wasn't as accurate as the neural networks, but it was much faster to train.
The Loser: GPR trains very fast but trails the neural networks in accuracy, while Random Forest was the weakest performer overall.

The Trade-Off: Speed vs. Accuracy

The researchers also looked at the "cost" of training these models:

FT-Transformer: Took about 6 minutes to train, but once trained, it predicts results in 0.33 milliseconds. This is millions of times faster than the old "master chef" (FEA).
CatBoost: Took only 37 seconds to train and offered a very good balance of speed and accuracy.
The Old Way (FEA): Takes minutes or hours per single test.

The Bottom Line

This study proves that for designing these new, rare-metal-free electric motors, Deep Learning models (specifically FT-Transformer) are the best tools available. They are incredibly accurate and can explore thousands of design possibilities in the time it used to take to test just one.

If you need the absolute best design, use the FT-Transformer. If you need a quick, "good enough" answer for early planning, CatBoost is a solid, fast alternative. Either way, they both make the design process dramatically faster than the traditional methods.

Technical Summary: Comparative Evaluation of Machine Learning and Deep Learning Models for WRSM Performance Prediction

Problem Statement
The rapid expansion of the electric vehicle (EV) industry has intensified the demand for high-efficiency electric motors that avoid reliance on rare-earth elements (REE). Wound-rotor synchronous motors (WRSM) have emerged as a strategic alternative to permanent magnet synchronous motors (PMSM) due to their ability to eliminate REE dependency. However, the design of WRSMs involves a high-dimensional, nonlinear optimization problem requiring the simultaneous tuning of numerous geometric and electromagnetic parameters. Traditional design validation relies on Finite Element Analysis (FEA), which, while accurate, is computationally expensive, often requiring minutes to hours per evaluation. This bottleneck severely restricts the exploration of large parameter spaces during early design stages. While machine learning-based surrogate modeling has been proposed to accelerate this process, existing literature often limits comparisons to a narrow set of models, excludes modern deep learning architectures, and lacks comprehensive benchmarks specific to WRSMs.

Methodology
This study conducts a systematic comparison of eight machine learning and deep learning models across four algorithmic families: gradient boosting, ensemble, neural networks, and probabilistic methods. The evaluation focuses on predicting two critical performance metrics: torque and motor efficiency.

Dataset Generation: A dataset of 3,351 valid samples was generated using Latin Hypercube Sampling (LHS) within the Ansys Motor-CAD simulation environment. The dataset maps 23 input parameters (15 geometric and 8 electrical) to the target variables. The data was split into 70% training, 15% validation, and 15% testing sets.
Models Evaluated:
- Gradient Boosting: CatBoost, LightGBM, and XGBoost.
- Ensemble: Random Forest.
- Neural Networks: Multilayer Perceptron (MLP), Physics-Informed MLP (PI-MLP, which integrates electromagnetic constraints into the loss function), and FT-Transformer (a transformer architecture designed for tabular data using self-attention).
- Probabilistic: Gaussian Process Regression (GPR), trained on a subset of 400 samples due to computational complexity.
Training Protocol: To ensure robustness and reproducibility, each model was trained using 10 different random seed values. Hyperparameters were systematically optimized using the Optuna framework. All models were evaluated using $R^2$ , Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE).

Key Contributions
The study distinguishes itself from prior work through three primary contributions:

Broad Model Spectrum: It includes a comprehensive range of algorithms, notably integrating recent deep learning architectures for tabular data, specifically the FT-Transformer, which had not been previously benchmarked for WRSM surrogate modeling.
Rigorous Reproducibility: The use of a multi-seed protocol (10 seeds) and systematic hyperparameter optimization provides a statistically robust assessment of model performance, moving beyond single-run evaluations.
Multidimensional Analysis: The study employs a Pareto analysis to evaluate the trade-off between computational cost (training time) and prediction accuracy, offering practical guidance for different stages of the design process.

Results
The experimental results demonstrate a clear performance hierarchy:

Accuracy: Neural network-based models systematically outperformed tree-based models. The FT-Transformer achieved the highest single-model accuracy, with an overall $R^2$ of 0.9928 ± 0.0009. Specifically, it attained an $R^2$ of 0.9914 for torque prediction (MAE = 1.227 Nm) and 0.9942 for efficiency prediction (MAE = 0.728%).
Comparison: Tree-based models (CatBoost, LightGBM, XGBoost, Random Forest) yielded lower accuracy, with $R^2$ values ranging from 0.80 to 0.95. Among them, CatBoost performed best, while Random Forest exhibited the highest error.
Physics-Informed Approach: The inclusion of physical constraints in the PI-MLP did not yield a performance improvement over the standard MLP; in fact, the standard MLP slightly outperformed the PI-MLP in both metrics.
Computational Trade-off:
- FT-Transformer: Required the longest training time (378 seconds) but delivered the highest accuracy.
- CatBoost: Emerged as a Pareto-optimal solution, achieving an $R^2$ of 0.9592 in approximately 37 seconds, offering a favorable balance between speed and accuracy.
- Inference Speed: Once trained, the FT-Transformer model generates predictions in 0.33 milliseconds, representing a speedup of several orders of magnitude compared to the minutes-long computation time of FEA.

Significance and Claims
The authors claim that this study represents the first comprehensive benchmark in the field of WRSM surrogate modeling that incorporates recent deep learning architectures like FT-Transformer. The findings suggest that the continuous function approximation capabilities of neural networks are better suited to capture the smooth, nonlinear electromagnetic transitions of WRSMs than the piecewise linear decision boundaries of tree-based models.

The study concludes that while FT-Transformer is the superior choice for applications requiring maximum accuracy, CatBoost serves as an efficient alternative for rapid prototyping and pre-screening stages where computational cost is a primary constraint. The developed surrogate models significantly reduce evaluation time, enabling the rapid exploration of large parameter spaces in the early design stages of electric motors. Future work is proposed to assess the generalizability of these models to different WRSM configurations and to explore the integration of physics-informed modeling with larger datasets.