On the Limits of Interpretable Machine Learning in Quintic Root Classification

Imagine you are a detective trying to solve a mystery: Can a computer, just by looking at a pile of numbers, figure out the secret mathematical rules that govern them?

The authors of this paper set up a specific puzzle to test this. They chose quintic polynomials (equations with a highest power of 5, like $x^5 + \dots$ ).

Here is the catch: For simpler equations (like squares or cubes), mathematicians have known the "secret formulas" for centuries. But for quintic equations, there is no general formula that humans have ever discovered. It's a mathematical dead end. This makes it the perfect test: If a computer can learn the rules here without being told the formula, it would be a huge breakthrough in "Artificial Intelligence" discovering math on its own.

The Experiment: The "Black Box" vs. The "Rule Book"

The researchers pitted two types of AI against each other:

The "Black Box" (Neural Networks): Think of this as a super-smart but mysterious apprentice. It can look at the numbers and guess the answer very well, but it keeps its reasoning inside its head. You can't see how it decided.
The "Rule Book" (Decision Trees): This is a transparent, logical student. It makes decisions using simple "If this, then that" rules. If it learns the answer, you should be able to read its notebook and understand the logic.

The Setup:
They fed both students raw numbers (the coefficients of the equations) and asked them to classify the equations based on how many real roots (solutions) they have.

The Results: A Tale of Two Students

1. The Black Box (Neural Network) did surprisingly well.
It got about 84% of the answers right just by staring at the raw numbers. It learned a pattern, but it was a "fuzzy" pattern. It was like a chef who can taste a soup and say, "This needs more salt," without knowing the exact chemical formula for saltiness.

2. The Rule Book (Decision Tree) struggled.
Without help, this student only got about 60% right. It was like trying to navigate a maze with a blindfold on. It couldn't find the path because the raw numbers were too messy and complex for simple "If/Then" rules to handle.

The Gap:
The mysterious Black Box was winning, but we couldn't see why. The transparent Rule Book was honest but failing.

The Twist: The "Magic Ingredient"

The researchers then tried a trick called Knowledge Distillation. They took the "smart" Black Box, looked at its answers, and tried to teach the "Rule Book" student to mimic those answers.

But here is the kicker: The Rule Book still couldn't figure it out on its own. It needed a hint.

The researchers gave the Rule Book a specific, human-engineered feature called Crit8.

The Analogy: Imagine you are trying to guess how many times a rollercoaster dips below ground. The raw numbers are just the blueprints of the track. The "Crit8" feature is like a human pointing out, "Look at the peaks and valleys! Count how many times the track crosses the horizon line."

Once the Rule Book was given this one specific hint (counting sign changes at critical points), it suddenly became a genius. It matched the Black Box's 84% accuracy and, best of all, it wrote down a simple, human-readable rule:

"If the sign changes happen more than 1.5 times, there are 5 real roots. If less, there are fewer."

The Big Discovery: Approximation vs. Truth

The most important finding of the paper is about what the computer actually learned.

The Black Box didn't learn the "Magic Ingredient" (the mathematical rule). Instead, it learned a geometric approximation.
- Analogy: Imagine trying to draw a perfect circle. The Black Box is like a robot that draws thousands of tiny straight lines to look like a circle. It works great if you stay close to the drawing, but if you zoom out or change the scale, the lines don't fit anymore. It learned a "shape" based on the data it saw, not the underlying law.
The Rule Book (when given the hint) learned the symbolic invariant.
- Analogy: This is like knowing the formula for a circle ( $x^2 + y^2 = r^2$ ). No matter how big or small the circle is, or where you put it, the rule holds true.

The Conclusion: AI Needs a Human Guide

The paper concludes with a sobering reality check:

AI is great at guessing, but it is terrible at "discovering" new math on its own.

Even though the Neural Network got the right answers 84% of the time, it didn't find the rule. It just built a complex map of the specific data it was trained on. To get a human-readable rule, a human still had to step in, realize what the "Magic Ingredient" (Crit8) was, and feed it to the computer.

In simple terms:
If you want an AI to discover a new law of physics or math, you can't just throw raw data at it and wait. The AI will just memorize the data's shape. You still need a human to provide the "lens" or the "hint" to help the AI see the underlying structure. The dream of a computer autonomously writing the next great math textbook is still a long way off.

1. Problem Statement

The central research question is whether Machine Learning (ML) models can autonomously discover interpretable, discrete mathematical rules from raw numerical data without human-engineered invariants or symbolic guidance.

The Benchmark: The authors use the classification of real-root configurations of quintic polynomials (degree 5).
The Challenge: While polynomials of degrees 2, 3, and 4 have known algebraic discriminants (symbolic formulas) that determine root types, the Abel-Ruffini theorem proves that no general solution in radicals exists for quintic polynomials.
The Goal: To determine if ML models can "rediscover" meaningful mathematical invariants (like the discriminant) from raw coefficients alone, or if they merely learn continuous, data-dependent geometric approximations of the decision boundary.

2. Methodology

Experimental Setup

Dataset: 40,000 quintic polynomials generated with coefficients sampled uniformly from $[-10, 10]$ .
Classes: Three classes based on the number of real roots: 5 real roots (Class 0), 3 real roots (Class 1), and 1 real root (Class 2).
Ground Truth: Roots were computed numerically using NumPy's eigenvalue method with a threshold of $10^{-10}$ .
Evaluation: 5-Fold Stratified Cross-Validation across 20 independent seeds. Metrics reported as Mean $\pm$ 95% Confidence Interval (Balanced Accuracy).

Models Evaluated

A diverse set of models was tested, including:

Interpretable: Decision Trees (CART), Logistic Regression, Symbolic Regression (PySR).
Black-box/Ensemble: Support Vector Machines (SVM), Random Forest, Gradient Boosting, XGBoost, Neural Networks (MLP).

Feature Engineering Strategy

To test the limits of autonomous discovery, the authors compared performance on:

Raw Coefficients: Only the polynomial coefficients ( $a_0$ to $a_5$ ).
Engineered Features: 63 features derived from classical polynomial theory, including:
- Sturm Sequences: Sign changes at specific points.
- Descartes' Rule: Sign changes in coefficients.
- Newton's Sums: Power sums of roots.
- Critical Points: Roots of the derivative $p'(x)$ and sign changes of $p(x)$ at these points (specifically a feature named Crit8).
- Hybrid Symbolic: Tschirnhaus invariants and algebraic combinations.

Knowledge Distillation Protocol

To interpret Neural Networks (NN), the authors employed a distillation framework:

Train an NN on the data.
Use the NN to generate predictions (soft labels) on the training set.
Train a Decision Tree to mimic these predictions.
Analyze the tree's structure and feature importance (via SHAP) to extract the "rules" the NN learned.

3. Key Results

A. Validation on Lower Degrees (2–4)

Symbolic Regression: Successfully discovered the discriminant rule for quadratics (97.3% accuracy) but failed to find interpretable rules for degrees 3, 4, and 5, showing a "Complexity Cliff."
Interpretable Models: When provided with the correct algebraic invariants (e.g., discriminant ratios), Decision Trees achieved 100% accuracy on degrees 2–4, proving the methodology works when the mathematical structure is known.

B. Quintic Classification (Degree 5)

Raw Coefficients Performance:
- Neural Networks: Achieved strong performance (84.3% ± 0.9%) using only raw coefficients.
- Decision Trees: Performed significantly worse (59.9% ± 0.9%), barely above the 33% random guessing baseline.
- Gap: A 24.4 percentage point gap exists between black-box and interpretable models when no features are engineered.
Impact of Feature Engineering (Crit8):
- When the Crit8 feature (number of sign changes in the sequence of polynomial values at ordered critical points) was added:
  - Decision Trees jumped to 84.2% ± 1.2%, matching the NN performance.
  - Neural Networks improved modestly to 89.9%.
- Distillation: The distilled Decision Tree successfully mimicked the NN (98.9% fidelity). SHAP analysis revealed Crit8 accounted for 97.5% of the decision structure.
- Extracted Rule: The tree learned a simple threshold rule:
  - Crit8 $\le$ 0.5 $\rightarrow$ 1 real root.
  - 0.5 < Crit8 $\le$ 1.5 $\rightarrow$ 3 real roots.
  - Crit8 > 1.5 $\rightarrow$ 5 real roots.

C. Robustness and Generalization (Stress Tests)

Out-of-Distribution (OOD): When coefficients were scaled up to 10 $\times$ training bounds, Invariant Decision Trees (using exact algebraic rules) maintained 100% accuracy. Neural Networks degraded significantly (e.g., dropping from 95.4% to 83.1% for quartics).
Data Efficiency: Invariant models reached near-perfect accuracy with ~50–100 samples. Neural networks required thousands of samples and did not plateau, suggesting they are learning complex geometric approximations rather than symbolic laws.
Noise Robustness: Both models degraded identically under Gaussian noise, indicating that the difficulty lies in the inherent sensitivity of polynomial roots to coefficient noise, not just model architecture.

4. Key Contributions

Demonstration of the "Interpretability Gap": Proved that while ML can achieve high predictive accuracy on complex mathematical problems (quintics) using raw data, it does not autonomously recover the underlying discrete symbolic rules.
Quantification of Feature Necessity: Showed that a single human-engineered feature (Crit8) bridges the performance gap between black-box and interpretable models, accounting for 97.5% of the learned structure.
Distinction between Geometric vs. Symbolic Learning: Provided empirical evidence (via OOD and data efficiency tests) that Neural Networks learn continuous, data-dependent geometric approximations of the decision boundary, whereas true mathematical invariants are scale-invariant and discrete.
Benchmarking Framework: Established a rigorous protocol for testing ML interpretability in structured mathematical domains, moving beyond simple pattern recognition to rule extraction.

5. Significance and Conclusion

The paper concludes that autonomous discovery of interpretable mathematical rules from raw data remains an open challenge.

Implication: High predictive accuracy does not equate to interpretability. Neural networks can approximate complex mathematical boundaries but store this knowledge geometrically, making it inaccessible as explicit human-readable logic without human intervention (feature engineering).
Future Direction: To achieve true "AI for Science" in mathematics, future systems may require inductive biases that explicitly favor symbolic structures (e.g., neuro-symbolic approaches, equivariant networks, or advanced symbolic regression) rather than relying purely on data-driven approximation.

In essence, the study suggests that in structured mathematical domains, interpretability requires explicit structural bias, not just more data or larger models.