Technical Summary: CADS – Conformal Adaptive Decision System for Cost-Efficient Image Classification

1. Problem Statement

The widespread adoption of high-capacity AI models is hindered by a "one-size-fits-all" deployment strategy that ignores the inherent heterogeneity of sample complexity. In domains like medical imaging, computational resources are often wasted on "easy" routine cases by applying monolithic, heavy models indiscriminately. This approach creates a dual crisis:

• Environmental: The carbon footprint of training and deploying massive models is becoming unsustainable.
• Operational: Rapid innovation leads to "AI model aging," where models are frequently replaced rather than reused efficiently.

Existing dynamic frameworks, such as early-exit mechanisms or confidence cascades, often rely on heuristic triggers (e.g., maximum softmax probability) that lack formal statistical guarantees and are prone to miscalibration, particularly under distribution shifts. There is a critical need for a framework that dynamically routes samples based on real-time complexity while providing mathematically rigorous uncertainty quantification to balance the cost-accuracy trade-off.

2. Methodology: The CADS Framework

The Conformal Adaptive Decision System (CADS) is a sequential multi-model algorithm designed to treat model deployment as a resource allocation problem. It routes samples through a cascade of models ranging from lightweight "Scout" models to high-capacity "Oracle" architectures.

2.1 Core Innovations

CADS relies on three primary mechanisms:

1. Conformal Prediction for Uncertainty Quantification: Instead of relying on soft labels or arbitrary confidence thresholds, CADS constructs a prediction set $C(x)$ using Adaptive Prediction Sets (APS). The size of this set serves as a rigorous proxy for difficulty:

   • Singleton ($|C(x)|=1$): High certainty.
   • Binary ($|C(x)|=2$): Moderate uncertainty.
   • Difficult ($|C(x)| \ge 3$): High uncertainty.
     This allows the system to trigger expert consultation only when statistically necessary.

2. Complementarity Analysis: The system precomputes failure patterns between experts on a calibration set. It quantifies how well one expert $B$ can correct the errors of another $A$, both globally and specifically for class-pair confusions. This enables dynamic selection of the expert most proficient at resolving specific uncertainties.

3. Two-Level Weighted Ensemble: Final predictions are aggregated using a hybrid weighting scheme:

   • Global Weight ($w_{global}$): Based on the expert's overall accuracy.
   • Local Weight ($w_{local}$): Based on the expert's specific proficiency in the suspected class.
     The final ensemble probability is a linear combination of these weights.

2.2 Adaptive Exit Logic

The inference process halts when a predefined confidence threshold is met. The threshold $\alpha_{final}$ is dynamically adjusted based on:

• Consensus Boost: If a high percentage of consulted experts agree, the required confidence is lowered.
• Class Difficulty: The threshold is refined based on the historical difficulty of the consensus class.
  The cascade terminates if the ensemble confidence exceeds $\alpha_{final}$, the two most recent experts agree, and a minimum number of experts have been consulted.

2.3 Optimization

Hyperparameters (including the conformal non-coverage rate $\zeta$, confidence thresholds, and weighting exponents) are optimized using Bayesian optimization (Tree-structured Parzen Estimator) to maximize accuracy while strictly adhering to a computational budget constraint ($B$ in GFLOPs).

3. Experimental Setup

• Models: A pool of experts was categorized into three tiers:
  • Scouts: Lightweight models (e.g., MobileNetV3, EfficientNet-Lite0) for rapid complexity assessment.
  • Specialists: Mid-tier models (e.g., MobileViT, ConvNeXt V2 Atto) for higher accuracy.
  • Oracles: Heavy models (e.g., Swin V2, DINOv2 ViT-Base) for high-uncertainty cases.
• Datasets:
  • PathMNIST: A medical histology benchmark (9 classes) to test cost-efficiency on routine tasks.
  • CIFAR-100: A high-entropy, 100-class environment to test routing accuracy in complex label spaces.

4. Key Results

Evaluation on both datasets demonstrated that CADS consistently outperforms individual experts and conventional ensemble baselines at a fraction of the computational cost.

• PathMNIST: CADS achieved the target accuracy using approximately 12 times less computation than standard cumulative approaches. The system predominantly utilized Scout and Specialist models, reserving Oracles only for high-uncertainty cases.
• CIFAR-100: CADS surpassed the accuracy of the best-performing single expert (89.66% vs. 88.77%), demonstrating that adaptive routing effectively mitigates individual model biases.
• Ablation Studies:
  • Conformal Routing: APS-based routing outperformed standard uncertainty measures (Max Softmax, Entropy, Margin) in accuracy.
  • Component Necessity: Removing conformal exit logic made the system computationally infeasible under strict budgets. Adding complementarity routing and hybrid weighting further increased accuracy from 84.36% to 88.26%.
  • Weighting Strategy: The 0.6/0.4 hybrid weighting significantly outperformed uniform averaging, particularly at low budgets.

5. Significance and Claims

The paper claims that CADS represents a significant step toward sustainable and efficient AI deployment. Its primary contributions are:

• Mathematical Rigor: By replacing heuristic triggers with conformal prediction, CADS provides formal statistical guarantees for uncertainty quantification, which is essential for high-stakes decisions like medical diagnosis.
• Resource Efficiency: It drastically reduces the economic and environmental footprint of AI by accurately routing samples based on real-time complexity, avoiding the waste of heavy-model inference on routine cases.
• Collaborative Superiority: The system proves that a collaborative ensemble can outperform even the strongest single expert by intelligently leveraging model complementarity.

The authors position CADS not merely as an optimization technique, but as a foundational methodology for shifting from monolithic architectures to adaptive, multi-expert systems that balance diagnostic reliability with computational sustainability.