HyperANFIS: Enhancing Rule Representation and Interpretability in Adaptive Neuro-Fuzzy Systems via Hyperbolic Geometry

Technical Summary: HyperANFIS

Problem Statement
The Adaptive Neuro-Fuzzy Inference System (ANFIS) is a widely used interpretable framework that combines neural network learning with fuzzy IF-THEN reasoning. However, conventional ANFIS architectures operate within Euclidean space, where rule antecedents are constructed by conjoining univariate membership functions independently. This Euclidean formulation faces significant limitations:

1. Representational Capacity: As the number of input variables increases, the feature space becomes crowded and constrained, leading to a combinatorial explosion in the number of rules required to cover the data.
2. Structural Bias: Euclidean geometry provides a weak inductive bias for data exhibiting hierarchical, tree-like, or highly non-uniform structures. It fails to naturally capture the expanding representational capacity required by fine-grained hierarchical branches, which are common in real-world domains like healthcare and industrial systems.
3. Rule Quality: These geometric constraints can degrade model performance and limit the credibility and distinctiveness of the generated fuzzy rules.

Methodology
To address these limitations, the authors propose HyperANFIS, a neuro-fuzzy model that reformulates the ANFIS architecture within a hyperbolic manifold. The model preserves the scalar fuzzy semantics and the core IF-THEN structure of ANFIS while performing all critical inference steps in hyperbolic space. The architecture consists of five functional layers:

1. Hyperbolic Mapping: Standardized inputs and trainable rule centers (tangent centers) are mapped from a shared origin tangent space to a hyperbolic representation (either Lorentz or Poincaré models) using exponential maps. This allows for the representation of rule prototypes as global geodesic points.
2. Geodesic Fuzzification: Instead of independent coordinate-wise memberships, rule antecedents are defined as geodesic regions around global rule prototypes. The activation strength of a rule is derived from the intrinsic geodesic distance between the sample and the rule prototype, normalized by a learnable intrinsic scale.
3. Normalization: Log-memberships are normalized across rules to determine firing strengths, ensuring a global geodesic fuzzy rule rather than a product of independent dimensions.
4. Intrinsic Consequent Construction: Rule consequents are constructed using the intrinsic local coordinates of the sample relative to the rule prototype. These are mapped back to the output manifold.
5. Fréchet Mean Aggregation: The final output is computed by aggregating the manifold-valued consequents using a firing-weighted Fréchet mean (a generalization of the mean in curved spaces). For classification, class logits are derived from the squared geodesic distance to class prototypes; for regression, the output is decoded in the origin tangent chart.

Key Contributions

• HyperANFIS Model: The proposal of a geometry-aware neuro-fuzzy model that learns global IF-THEN rules on a hyperbolic manifold, enabling accurate predictions based on these rules.
• Unified Hyperbolic Framework: A unification of rule matching, local consequent construction, and consequent aggregation entirely within hyperbolic space, offering a new perspective on mitigating the geometric limitations of Euclidean ANFIS.
• Empirical Validation: Demonstration through experiments on multiple real-world datasets that HyperANFIS outperforms standard ANFIS and various variants while generating higher-quality, more credible fuzzy rules.

Experimental Results
The authors evaluated HyperANFIS against conventional ANFIS and four representative variants (FSRE-AdaTSK, FCM-ANFIS, IT2-ANFIS, and PSO-ANFIS) across five datasets: Spambase, Car, Zoo, WDBC (Wisconsin Diagnostic Breast Cancer), and NSL-KDD.

• Performance: HyperANFIS consistently achieved the best performance across all datasets and metrics (Accuracy, Macro-F1, and Recall). On average, it improved accuracy by 0.0473, Macro-F1 by 0.0706, and Recall by 0.0638 compared to the standard ANFIS baseline.
• Dataset Specifics: Improvements were particularly pronounced on the Zoo dataset (hierarchical taxonomic structure) and WDBC (complex, non-uniform diagnostic relationships), with accuracy gains of 11.11% and 7.02% respectively.
• Interpretability Analysis (WDBC Case Study):
  • Rule Distribution: Unlike conventional ANFIS, which exhibited a "winner-take-all" pattern where a single rule dominated (42.98% of samples), HyperANFIS distributed dominant coverage across five principal rules, fostering multi-rule participation.
  • Class Representation: Conventional ANFIS learned a highly imbalanced set of prototypes (11 benign vs. 1 malignant), whereas HyperANFIS learned a more coherent distribution (7 benign vs. 5 malignant).
  • Rule Collaboration: The mean off-diagonal cosine similarity between rule activation vectors increased from 0.006 (conventional) to 0.432 (HyperANFIS), indicating that HyperANFIS rules establish correlated yet differentiated relationships, better conforming to fuzzy reasoning principles.
  • Effective Rule Count: The mean entropy-derived effective number of active rules increased from 1.14 to 6.67, demonstrating a shift from single-rule dominance to a collaborative inference process.

Significance
The paper claims that HyperANFIS demonstrates that reconstructing ANFIS in hyperbolic space is both methodologically feasible and effective. By leveraging the exponential volume growth of hyperbolic geometry, the model can organize non-uniform and hierarchical rule relationships more effectively than Euclidean counterparts. The significance of this work extends beyond simple performance gains; it provides a new geometric perspective for refining the structure of ANFIS and other intrinsically interpretable models, suggesting that geometric alternatives to Euclidean space can enhance both predictive accuracy and the credibility of interpretable rule systems.