🤖 machine learning

Evidential Rule Learning for Interpretable Classification with Abstention

This paper introduces Fast Evidential Rule Learning (FERL), a method that generates interpretable fuzzy rule models capable of providing evidential outputs, smooth Lipschitz stability, and reliable abstention in a single deterministic pass, while achieving state-of-the-art accuracy and robust out-of-distribution detection performance across diverse benchmarks.

Original authors: Javier Fumanal-Idocin, Javier Andreu-Perez

Published 2026-08-07

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Javier Fumanal-Idocin, Javier Andreu-Perez

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Art of Knowing When You Don't Know

Imagine you are walking through a dense forest, and you come across a signpost. A standard signpost points you in one direction with absolute confidence: "Go Left." But what if the path is foggy, or the sign is cracked? A truly wise guide wouldn't just guess; they would say, "I'm not sure, maybe go left, maybe right, or maybe I should just stop and tell you I can't decide." This is the heart of a growing field in computer science called interpretable machine learning. For years, computers have been great at making predictions, but they often act like "black boxes"—they give an answer without explaining why, and they rarely admit when they are confused.

To fix this, scientists are building models that are transparent, like a clear glass box where you can see the gears turning. A key concept here is evidence. Think of evidence not as a single vote, but as a collection of clues. If you have a pile of clues pointing to "Left," you are confident. If the clues are mixed, you are uncertain. Another vital idea is abstention: the ability for a model to say, "I don't know," rather than guessing wrong. This is crucial for real-life situations, like medical diagnosis or self-driving cars, where a confident mistake is far worse than a cautious pause. The question this paper tackles is simple but hard: Can we build a computer model that is easy to understand, makes accurate guesses, and knows exactly when to throw up its hands and say, "I need more info"?

The "Fast Evidential Rule Learner" (FERL)

Enter FERL (Fast Evidential Rule Learning), a new method introduced by Javier Fumanal-Idocin and Javier Andreu-Perez. Imagine a detective trying to solve a mystery. Old-school detectives (like standard decision trees) might look at a clue and immediately shout, "It's the butler!" even if the evidence is shaky. Other modern detectives use complex, invisible neural networks that are great at guessing but terrible at explaining how they reached that conclusion.

FERL is like a detective who keeps a very organized notebook. It uses fuzzy rules, which are like flexible "if-then" statements. Instead of a hard "If the temperature is above 30°C," it says, "If the temperature is kind of hot." As the detective follows a path through their notebook (a tree of rules), they collect clues. Here is the magic trick: FERL doesn't just collect clues to make a guess; it collects clues to measure how much they know.

In this system, every clue has a "firing strength." If a clue is strong, it adds a lot of weight to a specific suspect (a class label). If a clue is weak or missing, it adds weight to "ignorance"—the idea that we simply don't have enough information yet. When the detective combines all the clues, they don't just get a single answer. They get a belief (how sure they are), a plausibility (how possible it is), and a set of possibilities. If the clues are too messy, the "ignorance" becomes so heavy that the detective decides to abstain and say, "I can't decide."

The paper argues against the common practice of "post-hoc calibration." This is like taking a finished, confident prediction and trying to slap a "confidence score" on it afterwards. The authors show that this is often unreliable, especially when the computer encounters something new (like a cat in a world where it only saw dogs). Instead, FERL builds the confidence and the ability to abstain directly into the rules as it learns. It's like building a car with brakes that are part of the engine, rather than trying to add them on later.

What FERL Found

The researchers tested FERL on 30 different datasets (collections of data with numbers and categories, like medical records or sports stats). They found that FERL is not only easy to read but also incredibly accurate.

Accuracy: The "deep" version of FERL achieved an average accuracy of 83.23% across these 30 datasets. This was statistically better than almost every other rule-based method they tested, including classic tools like CART and C4.5.
Size: Despite being so accurate, FERL was surprisingly small. The "compact" version used fewer than 6 rules to get 77.77% accuracy, while a standard tree needed over 250 rules to get a similar score.
The "I Don't Know" Superpower: When the data was confusing or outside what the model had seen before (Out-of-Distribution), FERL didn't just guess wrong. It correctly identified the confusion. In tests where it had to detect "novel" data it had never seen, FERL achieved an AUROC of 77.7, which is just as good as the most specialized, complex detectors built just for that job.
Set Predictions: Instead of guessing one answer, FERL often gives a "set" of likely answers (e.g., "It's either a cat or a dog"). When they measured how useful these sets were, FERL scored 0.80 (at a standard discount level), beating other methods that either guessed too confidently or were too vague.

The paper also tested FERL on image data, but not by looking at pixels directly. Instead, they used a system that first described the image using human concepts (like "has wings" or "is red") and then used FERL to make the final decision. In this setup, FERL could not only classify the image but also point out which attributes were weird or anomalous. For example, if a picture of a bird had "wings" but "no legs," FERL could flag that specific contradiction.

The Bottom Line

FERL proves that you don't need a giant, confusing black box to get smart, reliable results. By using fuzzy logic to turn every rule into a piece of evidence, the model naturally learns to be confident when it should be, and to say "I don't know" when the evidence is thin. It is fast, it is small, and it is honest about its own uncertainty. The authors suggest that this approach offers a way to build AI that is not just accurate, but also trustworthy and transparent, capable of explaining its decisions and knowing its own limits without needing extra, complicated add-ons.

Technical Summary: Evidential Rule Learning with Abstention (FERL)

1. Problem Statement

Interpretable classification models, such as decision trees and rule lists, are essential for high-stakes domains where decisions must be justified. However, standard tree-based classifiers (e.g., CART, C4.5) often suffer from overconfidence, providing hard class labels or local distributions without a reliable account of the evidence supporting the decision. Furthermore, they lack mechanisms to abstain from making predictions when evidence is insufficient.

Existing approaches to improve reliability face significant limitations:

Post-hoc Calibration: Methods like temperature scaling or isotonic regression adjust probabilities but do not quantify the underlying evidence or support principled abstention. They often degrade under distribution shifts.
Uncertainty Estimation: Sampling-based methods (e.g., MC-dropout) require repeated inference, while deterministic uncertainty methods (DUMs) often rely on neural architectures that require spectral fine-tuning and may fail to detect "near-OOD" inputs that overlap with the training manifold. Additionally, these methods typically output scalar uncertainty scores without symbolic explanations for why a case is doubtful.
Credal Classifiers: While some tree-based credal classifiers exist, they often rely on imprecise leaf-frequency estimates, leading to miscalibration, or require complex post-hoc adjustments.

The paper addresses the need for a model that is simultaneously interpretable (rule-based), evidential (providing belief, plausibility, and ignorance), and capable of abstention, all within a single deterministic pass without auxiliary heads or held-out calibration sets.

2. Methodology: Fast Evidential Rule Learning (FERL)

FERL is a fuzzy rule-tree learner that integrates Dempster–Shafer (DS) theory directly into the rule-learning process.

2.1 Fuzzy Tree Construction

FERL grows a classification tree where internal nodes assign a feature and a fuzzy set.

Fuzzy Splits: Parameters for splits are learned using a bootstrapping procedure to ensure statistical robustness. For deeper trees, the algorithm finds Gini-optimal cuts in resamples to determine a fuzzy partition defined by a median threshold ( $\theta$ ) and a robust spread measure ( $h$ ).
Bounded Support: To prevent false confidence on out-of-distribution (OOD) data, the support of fuzzy membership functions is soft-bounded to the feature range observed during training. Points falling outside this range have their firing strength decay to zero, increasing the model's "ignorance."
Geometry Awareness: To detect near-OOD inputs (novel classes overlapping the training manifold), FERL models unsplit features at every node using a diagonal Gaussian weighted by the firing strength of the path. This allows the identification of specific anomalous attributes.

2.2 Evidential Output Generation

In FERL, every activated node $o$ emits a DS mass function on the frame of discernment $\Theta$ :

Mass Assignment: The firing strength $\phi_o(x)$ acts as a reliability discount. The node places mass $\phi_o(x) p_o(c)$ on class $c$ and residual mass $1 - \phi_o(x)$ on the entire frame $\Theta$ (representing ignorance).
Combination: Active nodes are combined using Dempster's rule of combination.
- In compact trees, internal nodes and leaves are combined.
- In deep trees, only leaves are combined to avoid the "dependence problem" where correlated evidence from nested nodes drives ignorance to zero artificially.
Prediction:
- Point Prediction: Derived from the pignistic transform of the combined mass.
- Set-Valued Prediction: Derived via interval dominance ( $Pl(c) \ge \max_{c'} Bel(c')$ ). If the prediction set covers the entire frame $\Theta$ , the model abstains.
- Analytical Form: The paper derives a closed-form expression for the combined mass, showing that repeated combination monotonically shrinks ignorance when evidence is non-vacuous.

2.3 Stability Analysis

The authors prove that FERL is Lipschitz stable. The unnormalized evidential masses are Lipschitz continuous with respect to the input, with a constant dependent on the tree depth ( $D$ ), the number of combined nodes ( $K$ ), and the maximum slope of fuzzy memberships ( $\lambda$ ). This ensures that small input perturbations result in smooth variations in evidential outputs.

3. Key Contributions

Native Evidential Rule Learner: FERL is the first model to function as an interpretable rule learner, a credal classifier with abstention, and an OOD detector in a single deterministic pass. It computes belief, plausibility, and abstention directly from fuzzy memberships without auxiliary heads or repeated inference.
Closed-Form Characterization: The paper provides an analytical account of the tree construction and evidential prediction, including a local stability certificate controlled by fuzzy band widths and a proof that nested node combination monotonically shrinks ignorance.
Broad Empirical Evaluation:
- Tabular Data: Benchmarked on 30 diverse datasets.
- Image Data: Integrated into Concept Bottleneck Models (CBMs) on CUB and AwA2 datasets, operating on concept vectors to provide human-understandable attribute-level diagnostics.

4. Experimental Results

4.1 Tabular Classification

Accuracy vs. Complexity: On 30 tabular datasets, FERL-deep (200 rules) achieved 83.2% average accuracy, statistically significantly outperforming other rule learners (e.g., CART at 79.8%, NeuRules at 79.3%) and matching the performance of non-interpretable ensembles like Random Forest (86.6%) and Gradient Boosting (85.6%) more closely than other interpretable methods.
Efficiency: FERL-deep trains in 0.26s and infers in 1.5ms, significantly faster than other evidential or neural rule learners (e.g., FUCS takes 140s to train).
Set-Valued Prediction: FERL-deep achieved the best utility-discounted accuracy ( $u_{65}=0.80, u_{80}=0.83$ ) among credal classifiers, with a high set coverage (0.92) and smaller average set sizes compared to conformal baselines.
OOD Detection: FERL-deep matched dedicated OOD detectors (77.7 AUROC vs. 77.4 for kNN distance) on leave-one-class-out tasks, while uniquely identifying which attributes were anomalous.

4.2 Concept Bottleneck Integration

Accuracy: In CBM settings, FERL-deep (with per-concept calibration) achieved 86.28% on CUB-20 and 65.18% on CUB-200, outperforming CART and concept-restricted Logistic Regression.
OOD in Open-World: In detector-class-disjoint settings, FERL remained competitive with dedicated detectors. On AwA2, it achieved 86.2 AUROC (within 2.0 points of kNN) and the best AUPR-Out (68.3) and unseen-class rejection (57.2%), all while naming the specific anomalous attributes.

5. Significance and Claims

The paper claims that FERL successfully bridges the gap between high-performance classification and rigorous interpretability. By embedding evidential reasoning directly into fuzzy rule learning, FERL avoids the pitfalls of post-hoc calibration and the computational overhead of sampling-based uncertainty.

Reliability: The model provides a "native" mechanism for abstention and uncertainty quantification that is robust to distribution shifts and capable of detecting both geometric and attribute-level anomalies.
Interpretability: Unlike black-box uncertainty methods, FERL's abstention and uncertainty are explained by specific fuzzy rules and attributes, making it suitable for high-stakes domains where understanding why a model is uncertain is as important as the prediction itself.
Scalability: The method is computationally efficient, offering a viable alternative to complex neural uncertainty models for tabular and concept-based image data.

The authors conclude that a single FERL head can provide auditable rules, evidential abstention, and attribute-level novelty diagnostics without the need for an auxiliary uncertainty model.