🤖 machine learning

From Classification to Regression: Using a Fruitfly to Solve Equations

Motivated by fruitfly sensory mechanisms, this paper proposes a novel regression framework that replaces complex global models with a library of local patterns, enabling efficient and controllable predictions for nonlinear dynamical systems and physics-informed learning through similarity-based weighted reconstruction.

Original authors: Shady E. Ahmed, Panos Stinis

Published 2026-07-30

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Shady E. Ahmed, Panos Stinis

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a computer how to predict the future. Usually, we ask computers to learn a giant, all-encompassing rulebook—a single, massive formula that explains how the world works from start to finish. It's like trying to memorize every single possible move in a game of chess before you're allowed to make your first one. This works, but it requires a supercomputer's brain and takes forever to run. However, nature has a smarter way of doing things. Think about a fruit fly buzzing around your kitchen. It doesn't have a giant map of the whole house in its tiny head. Instead, it senses its immediate surroundings, recognizes a familiar "scent" or pattern (like the smell of rotting fruit), and reacts based on what it has learned about that specific spot. It doesn't need to know the physics of the whole house; it just needs to know what to do right now based on what it sees.

This paper, titled "From Classification to Regression: Using a Fruitfly to Solve Equations," explores a new way to teach computers to solve complex math problems by copying that fruit fly strategy. In the world of science, "regression" is just a fancy word for finding a relationship between inputs and outputs—like predicting how hot a metal bar will get based on how long you heat it. Usually, scientists try to build one giant, complicated model to do this. But this paper suggests that instead of one giant brain, we should use a small library of "patterns." Think of it like a choose-your-own-adventure book. Instead of writing a new story for every possible situation, you have a few key scenes (patterns) and a set of rules for how to mix them together to tell the story of the moment. The authors show that by treating the problem like a game of "which pattern does this look like most?" (classification) and then blending the answers, computers can solve difficult equations much faster and with less memory, all while staying surprisingly accurate.

The Fruitfly's Secret: Solving Equations by Smelling the Air

The authors, Shady E. Ahmed and Panos Stinis from Pacific Northwest National Laboratory, propose a clever trick: stop trying to learn the whole map and start learning the landmarks.

In traditional machine learning, if you want a computer to predict how a system changes over time (like the weather or the movement of planets), you usually train a massive neural network to act as a "global surrogate." This is a single, complex model that tries to understand the entire input space at once. The problem is, these models get huge, expensive, and slow. They are like trying to carry the entire library of Congress in your backpack just to find one book.

The authors argue that most scientific data doesn't actually fill up the whole space. A planet doesn't fly randomly through the universe; it follows a specific orbit. A fruit fly doesn't fly in a straight line through a wall; it follows the scent of food. These systems revisit the same "regimes" or behaviors over and over again. The paper suggests that instead of learning a global rule, we should build a finite library of representative local patterns.

Here is how the "Fruitfly Regressor" works, using the analogy of a scent:

The Library of Scents (Patterns): Imagine you have a library of "scents" (patterns). Each scent represents a specific, recurring behavior of the system. For a swinging pendulum, one pattern might be "swinging fast to the left," and another might be "slowly stopping at the top."
The Sniff Test (Similarity): When you have a new situation (a "query"), the computer doesn't try to solve the whole equation from scratch. Instead, it "sniffs" the situation. It compares the current state to all the scents in its library.
The Weighted Mix (Reconstruction): The computer doesn't just pick the one best scent. It calculates how much the current situation smells like each scent. If it smells 60% like "fast left" and 40% like "slow top," it mixes the known answers for those two patterns together (60% of answer A + 40% of answer B) to get the final prediction.

This approach turns a hard regression problem (predicting a continuous number) into a classification problem (figuring out which patterns are present) and then a simple math problem (mixing the answers).

What the Paper Actually Found

The authors tested this idea on two very different types of problems: Dynamical Systems (things that change over time, like planets or fluids) and General Regression (finding relationships in data).

1. The Fruitfly in Motion (Dynamical Systems)
They applied their method to the Lotka-Volterra system (a model of how predator and prey populations interact) and the Lorenz system (a famous model of chaotic weather patterns).

The Result: Even with a very small library of patterns (as few as 4 to 32 patterns), the model could predict the future behavior of these systems with high accuracy.
The Surprise: They found that the "boundaries" between these patterns naturally formed around the system's stable points (like the center of a storm). Even if a pattern wasn't the "winner" in a specific region, it still contributed to the final answer, making the prediction smooth and stable.
The "Unknown" Equations: In one experiment, they didn't even know the equations governing the system. They used a temporary "helper" model to learn the patterns, but once the patterns were found, they threw away the helper. The final model only needed to store the patterns and their responses. This is a huge win for memory, as the final model is tiny compared to the giant neural networks usually required.
Continual Learning: They showed that if the system changes (like the weather shifting to a new pattern), you can just add a few new "scent" patterns to the library without forgetting the old ones. This avoids the "catastrophic forgetting" problem where AI usually forgets old lessons when learning new ones.

2. The Fruitfly in the Lab (General Regression)
They also tested this on standard math problems, like fitting a curve to a parabola or solving the 1D Poisson equation (a physics equation describing how heat or electricity spreads).

The Setup: They treated the input variable (like a position on a line) as a "scent" and the patterns as fixed points with their own "scent clouds."
The Results:
- For a simple parabola, they used 10 patterns and got a test error of 0.1%.
- For a complex wave made of 10 different sine waves, they used 30 patterns and got an error of around 1%.
- For a tricky physics problem with a sharp corner (a parabola that is zero on one side and curved on the other), they used 100 patterns and got an error of around 10%. The paper notes this higher error is expected because the function has a sharp, non-smooth edge, and the "scent" spread wasn't perfectly tuned for that specific shape.
Speed: The paper highlights that solving for the pattern values took only 0.01 seconds on a standard MacBook for the simpler problems, and even the most complex one took only 3.5 seconds.

What This Means (and What It Doesn't)

The paper suggests that we don't always need to build a giant, all-knowing brain to solve complex problems. By breaking the problem down into a library of small, recognizable "patterns" and mixing them based on similarity, we can create models that are:

Smaller: They require far fewer parameters to store.
Faster: The online prediction step is just a similarity check and a weighted sum, not a massive calculation.
Flexible: They can be updated easily as new data arrives.

However, the paper is careful not to claim this is a magic bullet for everything. The authors explicitly note that their method relies on the data having "recurring behaviors" or local structures. If a system is completely random and never repeats a pattern, this library approach might struggle. Also, while the results are promising, they are based on simulations and specific mathematical examples; the paper does not claim to have solved real-world climate modeling or biological systems yet, though it suggests a path forward.

The authors also point out a few things they haven't fully figured out yet. For instance, they aren't sure exactly how to choose the "variance" (the spread of the scent) for the patterns. They found that setting the variance to the size of the domain divided by the number of patterns worked best, which felt counterintuitive (like expecting a random walker to move in a straight line), but it worked in their tests. They also suggest that the number of patterns could be made adaptive, growing only when the system gets too complex for the current library.

In the end, this paper offers a playful but powerful shift in perspective: instead of trying to memorize the whole ocean, just learn the currents. By mimicking the fruit fly's ability to navigate by recognizing local scents, we might just be able to solve the world's hardest equations with a much lighter touch.

Technical Summary: From Classification to Regression: Using a Fruitfly to Solve Equations

Problem Statement
The paper addresses the challenge of learning nonlinear input-output relationships in scientific computing, specifically for predicting nonlinear dynamical systems and solving regression tasks. Traditional approaches rely on global surrogate models (e.g., deep neural networks, Gaussian processes, kernel methods) that approximate the entire mapping with a single, globally parameterized function. While effective, these methods often require large model capacities, high inference costs, and significant memory, making them unsuitable for energy-constrained environments like edge computing. Furthermore, scientific data often occupy only limited, recurring regions of the input space, suggesting that a global approximation may be unnecessary. The authors propose that accurate prediction can be achieved not by learning a global map, but by identifying and combining a small number of representative local behaviors.

Methodology
The core contribution is a framework that reformulates regression as a classification problem, inspired by the mechanism fruitflies use to sense their environment. Instead of learning a global function, the method learns a finite library of "representative patterns" and their associated responses.

Pattern Library and Embedding:
- For Dynamical Systems: The system state $u(t)$ is compared against a library of representative patterns $\{v_k\}$ . The similarity is computed via an inner product.
- For General Regression: An embedding $\phi(x)$ maps independent variables into a feature space. Inspired by the fruitfly olfactory system, the method represents the "scent" of a query point and pattern points as Gaussian influence functions. These are sampled at discrete points to form vectors, which are then compared.
Classification and Reconstruction:
- Similarities between the query (or state) and the stored patterns are transformed into normalized activation weights using a softmax function.
- The output is reconstructed as a weighted combination of the stored responses associated with the patterns.
- Continuous-Time Dynamics: The framework approximates the governing operator $\dot{u} = f(u)$ by reconstructing $\hat{f}(u)$ as a convex combination of pre-computed responses $f(v_k)$ .
- Discrete-Time Dynamics: The framework learns a one-step flow map $M(u)$ by reconstructing state transitions $M(v_k)$ .
- General Regression: The target value $u(x)$ is predicted as a weighted sum of trainable pattern values $\tilde{u}_{P_k}$ .
Training and Supervision:
- Direct Operator Supervision: If the governing equations are known, pattern responses are pre-computed, and the pattern library is learned by minimizing the error between the true and reconstructed operators.
- Trajectory Supervision: If only data is available, a surrogate model (e.g., a neural network) is used to approximate responses during the offline phase. Once patterns are identified, the surrogate is discarded, and only the pattern library is retained for inference.
- Physics-Informed Learning: For problems without data (e.g., solving PDEs), the pattern values are optimized by minimizing a physics-informed loss (e.g., the residual of the Poisson equation) directly, treating the pattern values as trainable parameters.

Key Contributions

Reformulation of Regression: The authors introduce a general framework that treats regression as a weighted reconstruction over a finite set of representative patterns, effectively bridging classification and regression.
Offline-Online Decomposition: The method separates the computationally expensive learning phase (offline) from the inference phase (online). Online prediction requires only similarity evaluation and weighted aggregation, decoupling inference cost from the size of the original training dataset.
Unified Framework: The approach unifies dynamical system prediction, supervised regression, and physics-informed learning under a single computational structure, differing only in the construction of the input embedding.
Explicit Control: The framework offers explicit control over the trade-off between accuracy, storage, and inference cost by adjusting the number of stored patterns and the similarity metric.

Results
The paper validates the approach on several benchmarks:

Lotka-Volterra System: The framework successfully reconstructed system dynamics using varying numbers of patterns. It demonstrated that increasing pattern count monotonically reduced error. Notably, the method identified that pattern dominance regions partition the state space, with boundaries intersecting at the system's fixed point.
Lorenz System: Using only data (no governing equations), the method learned a flow map that accurately predicted chaotic trajectories over long time horizons. A continual learning strategy was demonstrated where new patterns were added to capture dynamics outside the initial training regime, significantly reducing error compared to a baseline model.
Data-Driven Regression: The method achieved low relative errors (0.1%–1%) on 1D function regression tasks (parabolas, sine combinations, discontinuous functions) using simple least-squares optimization to find pattern values.
Physics-Informed Regression: The framework solved the 1D Poisson equation with various forcing functions. By minimizing the PDE residual, the method achieved relative errors around 1% for smooth functions and 10% for non-differentiable forcing functions, with solution times of milliseconds to seconds on a standard laptop.

Significance and Claims
The authors claim that this approach offers a compact representation where complexity depends on the number of learned patterns rather than the size of the training data. Unlike mixture-of-experts models that require evaluating multiple complex sub-networks, this method relies on similarity-based selection of pre-stored responses. The paper emphasizes that this biomimetic strategy shifts the computational burden from repeated local regression to similarity-based pattern selection, making it suitable for scenarios where inference cost and memory are critical constraints. The work suggests a fertile interface between classification and regression, proposing that complex functions may be more efficiently represented locally through recurring patterns than globally.

The Fruitfly's Secret: Solving Equations by Smelling the Air

What the Paper Actually Found

What This Means (and What It Doesn't)

More like this