NestyNet. I. Physics Functions Are Hard to Fit with Neural Networks: A Framework for Accurate Surrogates and Analytic Derivatives

Imagine you are trying to teach a robot to predict the weather. You give it a million data points about temperature, wind, and pressure. Standard AI tools, like the ones used to generate funny cat pictures or write poems, are great at finding patterns in messy data. They are like a very talented artist who can sketch a pretty good picture of a storm just by looking at a few photos. But in physics, "pretty good" isn't enough. If you are trying to predict how a star moves or how a bridge holds up, you don't just need to know where the object is; you need to know exactly how fast it's speeding up, how the forces are twisting, and how the energy is flowing.

This is where the trouble starts. Physics equations are often like a jagged mountain range with deep, narrow valleys. Standard AI tools try to climb these mountains by taking small, random steps. They might get close to the bottom of a valley, but they often get stuck on the steep sides or miss the true bottom entirely. When they do get close, their guesses about the "slope" (how fast things are changing) are usually wrong. It's like trying to drive a car down a steep, winding road while wearing blindfolded goggles that only show you the road ahead, not the curves. You might stay on the road, but you'll crash the moment you need to turn sharply. Scientists need a way to navigate these tricky mathematical landscapes with perfect precision, not just a rough sketch.

This is the problem tackled by a new framework called NestyNet, introduced by Rodrigo Ibata and his team. They realized that the issue wasn't that the AI wasn't smart enough, but that the "map" it was using was too wobbly. To fix this, they built a new kind of AI that doesn't just guess the answer; it builds a mathematical structure that is guaranteed to be smooth and easy to measure.

Think of standard AI as a giant, flexible net made of rubber bands. You pull the net to fit the data, but the rubber bands stretch and twist in unpredictable ways, making it hard to know exactly how the net is curving. NestyNet, on the other hand, is like building a wall out of smooth, interlocking Lego bricks. Each brick is a simple, predictable curve. When you stack them together, the whole wall is still smooth, but because you know exactly how every single brick is shaped, you can instantly calculate the slope, the curve, and even the area under the wall without doing any messy math.

The team tested this new "Lego wall" approach on 120 famous physics equations, ranging from simple mechanics to complex quantum problems. The results were staggering. Compared to the standard "rubber band" AI, NestyNet was not just a little better; it was a giant leap forward. For the basic values, it was about 2,100 times more accurate. For the first derivatives (the slopes), it was 1,400 times more accurate. And for the second derivatives (how the slopes are changing), it was 780 times more accurate. Even when they tried to polish the standard AI's answers with extra math tricks, NestyNet still outperformed them by hundreds of times.

But the magic didn't stop at accuracy. Because NestyNet is built from these smooth, analytic bricks, it doesn't need to use the slow, heavy machinery that other AIs use to figure out slopes. It can calculate them instantly. In fact, it was up to 44 times faster than the standard methods when calculating these derivatives. This speed and precision make it a perfect tool for "Physics-Informed Neural Networks" (PINNs), where the AI has to solve complex equations like the heat equation or fluid dynamics in real-time.

The authors also discovered a fascinating "law" about how these errors behave. They found that if an AI fits the data points well and also controls how much it curves (its curvature), it almost automatically gets the slopes right. It's as if the math has a hidden rule: if you get the shape and the bend right, the speed is just the geometric average of the two. This allows scientists to trust the AI's predictions about how fast things are moving, even without having direct measurements of the speed.

To prove this wasn't just a game with made-up numbers, the team used NestyNet to solve a real-world astrophysical puzzle: measuring the vertical acceleration of stars in our Milky Way galaxy. By fitting the positions and speeds of millions of stars, NestyNet was able to calculate the invisible gravitational forces acting on them with high precision. This is a task that has challenged astronomers for a century, and NestyNet handled it by turning a noisy, messy dataset into a clean, reliable physical measurement.

In short, NestyNet is a new toolkit for scientists who need their AI to be not just a pattern-matcher, but a precise calculator. It replaces the wobbly rubber bands of standard AI with smooth, predictable Lego bricks, allowing researchers to solve the hardest, most "stiff" problems in physics with unprecedented accuracy and speed. It shows that when the math gets tough, sometimes the best way forward isn't to make the AI bigger, but to make its building blocks smarter.