📊 statistics

On Fibonacci Ensembles: An Alternative Approach to Ensemble Learning Inspired by the Timeless Architecture of the Golden Ratio

This paper introduces "Fibonacci Ensembles," a novel ensemble learning framework inspired by the Fibonacci sequence that utilizes normalized Fibonacci weights and a second-order recursive dynamic to achieve systematic variance reduction and enhanced representational depth, demonstrating competitive performance against classical aggregation methods in controlled regression experiments.

Original authors: Ernest Fokoué

Published 2026-08-18

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Ernest Fokoué

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of modern computing, machines often learn by looking at many different guesses and then combining them into a single, better answer. This approach, known as ensemble learning, is like asking a panel of experts for advice rather than relying on just one. If the experts are diverse and imperfect, their combined wisdom can often outperform any single individual. The challenge for scientists is figuring out how to mix these opinions. Should every expert get an equal vote? Or should some voices be louder than others? For decades, the standard practice has been to treat every learner the same, giving them equal weight, or to let the data decide the mix through complex, trial-and-error methods. But what if there were a natural, pre-set rule for this mixing that required no tuning and no guesswork?

A researcher at the Rochester Institute of Technology decided to test a very specific idea inspired by nature: the Fibonacci sequence. This is a famous pattern of numbers found in the spirals of shells and the arrangement of leaves, where each number is the sum of the two before it. The question was whether this ancient mathematical rhythm could serve as a perfect recipe for combining machine learning models. The researcher set out to see if weighting the experts according to this sequence would create a smarter, more stable prediction system than the usual methods. The investigation revealed a surprising truth about the limits of this approach and clarified exactly when such a pattern helps and when it hinders.

The study began by arranging a series of learning models in a specific order, typically from simple to complex, and then assigning them weights based on the Fibonacci numbers. The most recent and complex models received the largest weights, following the sequence's growth. The researchers then ran thousands of simulations to see how this method performed compared to giving every model an equal say. They discovered a hard limit on how many models this system could actually use. No matter how many experts were added to the group, the Fibonacci weighting scheme effectively listened to only about four of them. The weights for the older, simpler models became so tiny that they were practically ignored. This meant that adding more models to the mix did not make the system more stable or accurate, which is the main benefit of using large groups of learners in the first place.

Because the system effectively ignored most of the available experts, it could not reduce the random errors that come from having too few data points. In fact, for any group larger than four, the standard method of giving everyone an equal vote was mathematically proven to be better at reducing these random errors. The only way the Fibonacci method could possibly win was if the order of the models mattered deeply. If the models were arranged so that the most important ones were at the very end of the list, the Fibonacci rule would boost those specific voices. However, if the order was random or unimportant, the method would fail, often performing much worse than a simple average.

The researchers also tested whether the famous golden ratio, the number that the Fibonacci sequence approaches, was the perfect setting for this kind of weighting. They compared the Fibonacci rule against a wide range of other geometric patterns to see which one produced the most accurate results. The results were clear: the golden ratio was not the best choice. The ideal setting changed depending on the specific problem and the type of data. Sometimes the best pattern favored the simplest models, and other times it favored the most complex ones. The golden ratio was rarely the winner, and using it blindly could lead to errors that were many times larger than necessary.

To understand why this happened, the paper looked at the difference between two types of problems. In some cases, the data is like a set of building blocks where you need to pick the right few to build a wall; here, averaging everything works well. In other cases, the data is like a signal that fades out over time, where the early parts are strong and the later parts are weak noise. In these fading signal cases, the Fibonacci rule was actually quite bad because it naturally focused on the high indices (the complex models) regardless of where the actual signal energy was located. If the important information was concentrated in the early, simpler models, the Fibonacci rule upweighted exactly the wrong end of the spectrum. But this only worked if the models were arranged in the correct order. When the researchers shuffled the order of the models randomly, the performance of the Fibonacci method collapsed, swinging wildly between being very good and very bad. This proved that the method is not a magic bullet; it is a tool that only works if the user knows exactly how to arrange the tools.

The study also compared this approach to older, more sophisticated mathematical techniques used in statistics. They found that while the Fibonacci method was a decent, free-to-use option, it was still far behind the best possible methods that do not require the weights to add up to one. The main problem was not the Fibonacci pattern itself, but the rule that forced all the weights to sum to a specific total. This rule, which is common in many machine learning systems, acts as a bottleneck that prevents the system from being truly optimal. The Fibonacci sequence simply highlighted this limitation by showing how much information is lost when you force a group of experts to share a fixed budget of attention.

In tests using real-world data, such as the flow of the Nile River, sunspot activity, and ocean temperatures, the results confirmed the theory. On the sunspot data, where the patterns are clear and ordered, the Fibonacci method improved upon simple averaging. But on the Nile data, where the patterns were less structured, the method performed worse than just taking a simple average. The researchers found that the best way to set the weights was not to use a fixed number like the golden ratio, but to let the data itself decide the right mix through a process of testing and validation.

The final conclusion of the work is a modest but important correction to how we think about these systems. The Fibonacci sequence does not hold a secret key to perfect machine learning. Instead, it serves as a clear example of how a weighting rule acts as a filter. It shows that such a rule can only ever focus on a small, fixed number of models, no matter how many are available. The golden ratio, often seen as a symbol of perfect harmony in nature, turns out to be just one specific setting among many, and often not the best one. The real value of the study is in showing that the way we combine models is a critical part of the process that deserves careful design, rather than just being a default setting. The best approach is to understand the structure of the data and choose a weighting method that fits that structure, rather than hoping a single mathematical pattern will solve every problem.

Technical Summary: On Fibonacci Ensembles

Problem Statement
Ensemble learning relies on aggregating imperfect base learners to improve predictive performance. While methods like bagging and boosting are well-established, the aggregation rule itself—the weighting law applied to base learners—is rarely designed explicitly. It is typically inherited (uniform for bagging), implicit in the loss function (boosting), or estimated as a nuisance parameter (stacking). This paper investigates whether a specific, structured weighting law derived from the Fibonacci sequence can serve as a principled, tuning-free default for ensemble aggregation. The core question is whether the recursive architecture of the Fibonacci sequence, which governs growth in nature, offers a mathematically optimal strategy for combining learners, or if it merely imposes unintended constraints.

Methodology
The author defines Fibonacci weighting where base learners $h_1, \dots, h_M$ , ordered by increasing complexity, receive weights proportional to Fibonacci numbers $F_m$ :
$\alpha_m = \frac{F_m}{\sum_{j=1}^M F_j}$
The analysis proceeds through several theoretical and empirical lenses:

Asymptotic Analysis: Utilizing Binet's formula, the author derives the asymptotic behavior of these weights as $M \to \infty$ , showing they converge to a geometric progression with ratio $\phi^{-1}$ (where $\phi$ is the golden ratio).
Bias-Variance Decomposition: The risk of the ensemble is analyzed under an orthonormal basis assumption. The author derives an exact crossover condition determining when geometric weighting (including Fibonacci) outperforms uniform averaging, balancing bias reduction against variance inflation.
Recursive Flow Stability: The paper models ensemble construction as a second-order recursive flow ( $F_m = \beta F_{m-1} + \gamma F_{m-2} + \Delta F_{m-1}$ ). Using Schur–Cohn conditions, the author analyzes the stability of the classical Fibonacci recursion ( $\beta=1, \gamma=1$ ) and propose a "damped" variant that preserves the golden mode ratio while ensuring $L_2$ stability.
Comparative Benchmarks: The performance of Fibonacci and geometric weighting is compared against:
- Uniform averaging (standard baseline).
- Optimal geometric weighting (where the ratio $r$ is tuned).
- Classical ordered shrinkage estimators (Pinsker, Blockwise Stein) which do not enforce the simplex constraint ( $\sum w_m = 1$ ).
Experiments: Computational experiments utilize random Fourier features and polynomial dictionaries on synthetic targets ( $\sin(2\pi x)$ , $\text{sinc}(x)$ ) and real-world time series (Nile river flows, sunspot activity, El Niño sea-surface temperatures).

Key Contributions and Results

Fixed Effective Ensemble Size:
The central structural result is that normalized Fibonacci weights possess a fixed effective ensemble size ( $M_{eff}$ ) regardless of the total number of learners $M$ .
$M_{eff} = \left( \sum \alpha_m^2 \right)^{-1} \to \phi^3 = 2 + \sqrt{5} \approx 4.236$
Consequently, Fibonacci ensembles consult slightly more than four learners, even if thousands are available. This implies that Fibonacci weighting cannot achieve the $O(1/M)$ variance decay characteristic of bagging. For any $M > \phi^3$ , uniform weighting strictly dominates Fibonacci weighting in terms of variance.
Bias Reduction as the Sole Mechanism:
Because variance reduction is impossible, any advantage of Fibonacci weighting must stem from bias reduction. The paper establishes a necessary and sufficient condition (Theorem 3.1) for geometric weighting to outperform uniform averaging: the signal energy (coefficients $\theta_m^2$ ) must be concentrated on the indices that the weighting law upweights.
- If the signal is at low indices (smooth functions), the optimal ratio $r < 1$ (upweighting early learners).
- If the signal is at high indices, $r > 1$ .
- The Fibonacci ratio $\phi \approx 1.618$ corresponds to $r > 1$ , meaning it upweights the last two learners (the high indices). The paper finds that $\phi$ is rarely the optimal ratio; in many experiments, the optimal $r$ ranges from 0.20 to 2.20. Crucially, when the signal energy sits at low indices (a common scenario for smooth functions), the optimal $r < 1$ , making Fibonacci weighting (with $r=\phi > 1$ ) suboptimal because it upweights the "wrong end" of the dictionary.
Equivalence to Geometric Weighting:
The author proves that for $M \ge 20$ , Fibonacci weights are numerically indistinguishable from geometric weights with ratio $r = \phi$ . Thus, claims about "Fibonacci" weighting are effectively claims about the specific number $\phi$ .
Stability of Recursive Flows:
The classical Fibonacci recursion is shown to be unstable in an ensemble flow context ( $\rho(T) = \phi > 1$ ). The author constructs a damped Fibonacci flow that retains the structural property of the mode ratio ( $\lambda_+ / |\lambda_-| = \phi^2$ ) while satisfying Schur–Cohn stability conditions ( $\rho(T) < 1$ ).
The Constraint Limitation:
A critical finding is that the simplex constraint ( $\sum w_m = 1, w_m \ge 0$ ) is the primary bottleneck in ensemble performance, not the specific weighting law.
- Within the class of convex aggregation rules (simplex-constrained), the geometric family is nearly optimal (within 12% of the best possible simplex estimator).
- However, the entire class of simplex-constrained estimators is 6 to 146 times worse than ordered shrinkage methods (Pinsker, Blockwise Stein) which do not enforce $\sum w_m = 1$ . The constraint forces weights to average $1/M$ , causing severe underestimation of signal coefficients.
Order Dependence:
Structured weighting is highly sensitive to the ordering of the dictionary. Permutation studies show that randomizing the order of learners can degrade performance by factors of 2.5 to 15. Fibonacci weighting is only beneficial when the dictionary index carries genuine meaning (e.g., increasing complexity or frequency).

Significance and Claims
The paper draws a modest but precise conclusion: the Fibonacci sequence does not provide a "golden" optimal weighting law for ensemble learning.

The Golden Ratio's Role: $\phi$ is not a privileged weight; rather, $\phi^3$ represents the exact ceiling on the effective number of learners a Fibonacci scheme can utilize. It is a limit, not a solution.
The Real Problem: The weighting law is a "consequential and largely undesigned component" of ensemble learning. The paper argues that the aggregation rule should be estimated from data (e.g., via cross-validation) rather than assumed to be uniform or Fibonacci.
Practical Implication: While Fibonacci weighting is a tuning-free, interpretable default, it is not universally optimal. Its utility depends entirely on the bias-variance trade-off specific to the dictionary's ordering and the signal's spectral distribution. The paper emphasizes that distinguishing between "predictor-like" learners (where averaging is natural) and "basis-like" learners (where shrinkage is superior) is more important than the choice of a specific weighting sequence.

In summary, the paper reframes the Fibonacci ensemble not as a discovery of a universal law, but as a rigorous demonstration of the limits of convex aggregation and the critical importance of dictionary ordering and constraint selection in ensemble design.

More like this