AI Models Still Lag Behind Traditional Numerical Models in Predicting Sudden-Turning Typhoons
This study challenges the claim that AI-based weather models excel at predicting extreme events by demonstrating that while Pangu-Weather generally outperforms traditional numerical models in tracking tropical cyclones, it still lags behind in accurately forecasting rare, sudden-turning typhoon trajectories.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
For decades, the most reliable way to predict the weather has been to build a digital twin of the atmosphere. Scientists feed current observations into massive supercomputers that solve complex equations describing how air moves, how heat rises, and how water vapor turns into rain. These numerical models act like a physics engine, simulating the laws of nature step by step to forecast what will happen days from now. They are the backbone of modern forecasting, trusted because they are built on the fundamental rules that govern our planet. However, in the last few years, a new contender has emerged: artificial intelligence. Instead of solving equations, these AI models learn to predict the weather by studying millions of years of historical weather data, spotting patterns much like a student memorizing a textbook. They are incredibly fast and have shown great promise, leading many to believe they might soon replace the traditional, physics-based systems entirely.
A recent study by a team of researchers from China's meteorological and academic institutions puts this idea to a rigorous test, specifically looking at how well these AI models handle the most dangerous and unpredictable weather events: typhoons that suddenly change direction. The researchers focused on a specific type of storm behavior known as a "sudden turn," where a typhoon makes sharp, unexpected bends in its path, often catching forecasters off guard and leading to significant damage. While previous reports had celebrated AI's ability to predict extreme weather, those examples mostly involved storms that were extreme because of their sheer power, not because their paths were difficult to follow. This new analysis asked a harder question: can AI predict the rare, erratic movements that defy the usual flow of the atmosphere?
The team examined 104 typhoons in the Northwest Pacific between 2020 and 2024, sorting them into three groups: those that moved in a straight, predictable line; those that looped around; and those that made sudden, sharp turns. They compared the predictions of the leading AI model, known as Pangu-Weather, against the best traditional numerical models and the forecasts made by human experts from major meteorological agencies. The results were clear and nuanced. For the vast majority of storms, including those that moved normally or even those that looped, the AI model performed exceptionally well, often beating the traditional computer models in accuracy. It proved to be a powerful tool for capturing the general flow of the atmosphere.
However, the story changed completely when the researchers looked at the storms that made sudden turns. In these specific cases, the traditional numerical models consistently outperformed the AI. The study highlighted the 2023 Typhoon Khanun as a prime example. This storm made two sharp turns within five days as it passed near the Ryukyu Islands, a path that caused significant disruption. While the AI model was able to predict the storm's location accurately for the first day or two, its skill dropped off sharply as the forecast extended further. By the time the forecast reached five days, the AI's predictions were no better than those of human forecasters and were significantly less accurate than the traditional models. The error in the AI's path prediction for these sudden-turning storms was, on average, nearly 10 percent larger than that of the best traditional system.
The researchers dug deeper to understand why this gap exists. They found that the atmosphere usually guides typhoons with a steady, large-scale wind current, much like a river carrying a boat. When this current is strong and steady, the AI model, which is excellent at recognizing these large patterns, performs brilliantly. But when the steering current is weak, the storm's path is determined by a complex interaction between the storm itself and the surrounding air. In these moments, the fine details of the storm's internal structure become critical. The study suggests that because AI models are trained on historical data rather than the laws of physics, they struggle to simulate these intricate internal structures when the guiding winds are weak. They tend to produce smoother, more averaged-out predictions that miss the sharp, chaotic turns that occur in reality.
This does not mean the AI is a failure, but it does reveal a specific limitation. The traditional models, which are built to solve the physical equations of the atmosphere, are still better at capturing the physics of these rare, extreme events. The AI model, while faster and generally accurate, appears to lack the ability to "reason" through the complex physics required when a storm behaves unusually. The authors conclude that for the foreseeable future, the most reliable approach will not be to replace the old systems with the new ones, but to combine them. By integrating the physical constraints of traditional models into the training of AI, scientists hope to create a hybrid system that can handle both the common patterns and the rare, dangerous surprises of the weather. Until then, when a storm threatens to take an unexpected turn, the old-school physics-based models remain the most trustworthy guide.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.