Technical Summary: Neural Bayesian Sequential Routing (NBSR)

1. Problem Statement

Standard deep learning architectures rely on static, dense forward computations that process all input features simultaneously to produce a monolithic probability distribution. While effective for semantic accuracy, these models suffer from three critical limitations:

1. Lack of Cognitive Plausibility: They fail to model the sequential, evidence-accumulating nature of human decision-making, where hypotheses are progressively refined rather than evaluated all at once.
2. Poor Interpretability: The "black box" nature of dense networks obscures how evidence is acquired and how uncertainty evolves, making decision auditing difficult.
3. Inefficiency and Overconfidence: They expend unnecessary computational resources on easily classified inputs and often produce overconfident predictions on out-of-distribution (OOD) data due to the lack of native uncertainty quantification mechanisms.

While Mixture-of-Experts (MoE) architectures introduce conditional computation, they typically rely on soft, continuous routing that does not maintain a temporally evolving belief state, nor do they allow for sequential evidence accumulation or dynamic early exiting.

2. Methodology: Neural Bayesian Sequential Routing (NBSR)

NBSR is a dynamic framework that models neural inference as an active, evidence-accumulating traversal over a hierarchical Directed Acyclic Graph (DAG). It unifies conditional neural execution, evidential uncertainty, and sequential decision-making.

Core Architecture

• Global Knowledge Oracle: Unlike standard feed-forward networks where features are transformed and discarded layer-by-layer, NBSR embeds the input $x$ once into a dense feature vector $h_x$ using a shared backbone. This vector acts as a persistent "Knowledge Oracle" broadcast to all nodes in the DAG, ensuring every decision has access to the full contextual state.
• The Decision Graph: The model traverses a DAG where nodes represent decision states. At each node $v_t$, a Router Network (MLP) selects a discrete path based on the global features $h_x$ and the current Bayesian Belief State $\alpha_t$.
• Evidence Extraction: Upon reaching a selected node, a local Expert Network queries the Oracle $h_x$ to extract a strictly positive evidence vector $e_t$.
• Bayesian Belief Update: The belief state is updated via exact conjugate addition: $\alpha_{t+1} = \alpha_t + e_t$. Here, $\alpha$ represents the concentration parameters of a Dirichlet distribution. The evidence vectors $e_t$ act as Bayesian pseudo-counts, deterministically updating the Dirichlet state. This process mathematically enforces the "sharpening" of decision boundaries as evidence accumulates.

Training and Inference Mechanics

• Differentiable Discrete Routing: To enable end-to-end training of the discrete tree structure, NBSR employs the Gumbel-Softmax relaxation coupled with the Straight-Through Estimator (STE).
  • Forward Pass: The router selects a hard, single path (argmax), ensuring computational efficiency by only executing the visited nodes.
  • Backward Pass: Gradients flow through the continuous Gumbel-Softmax approximation, allowing the router to learn optimal path selection without the gradient vanishing problem of discrete operations.
• Training Objective: The loss function combines the Negative Log-Likelihood (NLL) of the final Dirichlet expectation with an entropy penalty term ($\lambda \sum H(\alpha_t)$). This encourages the model to not only maximize accuracy but also to reduce uncertainty (entropy) as rapidly as possible along the trajectory.
• Inference Strategies:
  • Early Exiting: If the differential entropy of the belief state $H(\alpha_t)$ falls below a threshold $\eta$, traversal halts immediately, saving computational FLOPs.
  • Epistemic Abstention: If the total precision $\alpha_0$ remains low (indicating high uncertainty) even after full traversal, the model abstains from prediction, providing a native safety mechanism for OOD inputs.

3. Key Contributions

1. Bayesian Sequential Routing Framework: A novel formulation where discrete routing decisions sequentially update a Dirichlet belief state via exact conjugate addition, mathematically enforcing progressive hypothesis sharpening.
2. End-to-End Hard Routing: Adaptation of the Gumbel-Softmax estimator to train discrete hierarchical decision trees, preventing representation collapse while enabling dynamic, entropy-based early exiting.
3. Theoretical Guarantees: Formal proofs demonstrating that under strictly positive evidence extraction, total Dirichlet precision increases monotonically along any valid trajectory, and marginal predictive variance is correspondingly bounded. The framework is shown to recover the Bayes-optimal conditional distribution under idealized capacity assumptions.
4. Interpretable and Safe Inference: The framework provides transparent, causal audit trails for every decision. It utilizes native Dirichlet precision to trigger OOD safety abstentions, preventing hallucinations.
5. Resource-Rational Extensions: Successful extension of the DAG topology to incorporate temporal memory (NBSR-Mem) for autonomous control and formulation as an Autoregressive State Machine (AR-NBSR) for cost-aware Bayesian Optimal Experimental Design.

4. Experimental Results

The framework was evaluated across five diverse domains:

• Visual Categorization (CIFAR-10): NBSR achieved a peak accuracy of 96.74%, outperforming both a Flat ResNet-18 (96.60%) and a Sparse MoE (96.43%). Crucially, it achieved a significantly lower Expected Calibration Error (ECE) of 0.015 compared to baselines (~0.027). The model demonstrated 90% early-exiting at Depth 1 without accuracy degradation, validating its resource-rationality.
• Structured Medical Diagnosis: In a clinical triage task, NBSR matched the diagnostic accuracy of a flat MLP (97.62%) while providing patient-specific, path-dependent feature attributions. It successfully reduced average inference depth from 2.0 to 1.0 for unambiguous cases, cutting inference time without performance loss.
• Language Modeling: On a controlled syntactic-semantic disambiguation task, NBSR preserved generative capacity (Perplexity 13.81 vs. 13.43 baseline) while exposing transparent routing traces (e.g., distinguishing syntactic vs. semantic experts). It demonstrated the ability to abstain on OOD prompts where standard softmax models hallucinate.
• Partially Observable Control (POMDP): In a navigation task with spatial amnesia, NBSR-Mem (incorporating recurrent memory) achieved a 100% success rate, matching the performance of a black-box CNN-GRU. Unlike the black-box baseline, NBSR-Mem successfully detected "Alien Hazard" OOD inputs and triggered a safe halt (100% abstention rate) rather than crashing.
• Active Clinical Triage (BOED): The AR-NBSR formulation acted as an active learning agent, learning to select diagnostic tests to maximize information gain per unit cost. It achieved peak diagnostic accuracy at a significantly lower average cost (~$45-$50) compared to greedy baselines ($60-$80), demonstrating non-myopic, resource-rational planning.

5. Significance and Claims

The paper posits that NBSR represents a fundamental shift from static, monolithic deep learning toward an active, sequential feature acquisition paradigm. Its significance lies in the unification of three previously separate concepts:

1. Conditional Execution: Moving from soft, parallel averaging to hard, path-dependent routing.
2. Evidential Uncertainty: Treating uncertainty not as an auxiliary diagnostic but as a state variable that actively controls computation (via early exiting and abstention).
3. Sequential Reasoning: Modeling inference as a forward-causal belief trajectory where intermediate states and expert contributions are inspectable.

The authors claim that NBSR provides a mathematically grounded framework for interpretable, modular, and resource-rational agentic AI. By ensuring that experts are selectively queried rather than softly averaged, and that uncertainty is intrinsic to the state, the framework offers a path toward AI systems that are not only accurate but also transparent, safe, and computationally efficient. The paper concludes that while NBSR is currently validated on specific benchmarks, its architecture naturally supports future extensions into neuro-symbolic integration, unbounded topologies via Dirichlet processes, and dynamic skill acquisition.