JANUS: Anatomy-Conditioned Gating for Robust CT Triage Under Distribution Shift

Technical Summary: JANUS – Anatomy-Conditioned Gating for Robust CT Triage

Problem Statement
Automated CT triage aims to flag clinically significant findings, many of which are defined by quantitative thresholds (e.g., organ size, vessel caliber, HU density) rather than visual appearance alone. While Vision Transformers (ViTs) provide strong visual representations, they often lack explicit mechanisms to incorporate these measurement cues. Under distribution shifts (e.g., changes in scanner protocols or reconstruction settings), visual appearance can vary significantly while underlying anatomy remains invariant. Previous attempts to bridge this gap using coarse organ-level statistics or additive fusion often fail to provide explicit suppression mechanisms, allowing models to latch onto spurious correlations when visual and quantitative evidence conflict.

Methodology: JANUS Architecture
The authors introduce JANUS, a physiology-guided dual-stream architecture designed to condition visual embeddings on macro-radiomic priors via Anatomically Guided Gating.

1. Data Ingestion & Preprocessing:

   • Visual Stream: 3D CT volumes are sampled into $N$ 2.5D "tri-slices" (stacking adjacent slices). These are processed by a pre-trained DINOv3 ViT backbone to extract patch embeddings.
   • Symbolic Stream: Segmentation masks (derived from tools like TotalSegmentator) are used to compute a disease-specific macro-radiomic prior vector ($s_\ell$). This vector includes descriptors such as organ volumes, diameters, densitometry profiles, and fluid volumes. These scalars are z-score normalized.

2. Visual Encoding (Organ-Masked Attention):

   • Instead of standard global pooling, JANUS employs Dilated ROI-Masked Attention Pooling. For each pathology, a specific 3D Region of Interest (ROI) mask is defined (e.g., the aorta for aneurysms).
   • This mask is dilated to capture peri-organ context and aligned with the tri-slice sampling.
   • Attention weights are computed only over tokens within the ROI, pooling visual features ($z_{v,\ell}$) specifically relevant to the pathology while ignoring irrelevant background.

3. Anatomically Guided Gating (The Core Mechanism):

   • The macro-radiomic prior vector $s_\ell$ is projected through a learned linear layer and passed through a sigmoid function to produce a physiological gate $g_\ell \in [0, 1]^d$.
   • This gate modulates the visual embedding via element-wise multiplication (Hadamard product): $z_{gated,\ell} = z_{v,\ell} \odot g_\ell$.
   • Mechanism: This acts as a "Physiological Veto." If quantitative priors contradict the visual evidence (e.g., a visual feature suggests an aneurysm, but the aortic diameter is within normal limits), the gate can attenuate specific dimensions of the visual embedding, effectively suppressing high-confidence false positives.
   • Initialization: The gate is initialized to remain mostly open ($g \approx 0.88$) to ensure stable gradient flow, allowing the model to learn stronger gating only when supported by training data.

4. Training Objective:

   • The model uses a curriculum-weighted Binary Cross-Entropy (BCE) loss over partially observed labels. Missing labels are treated as weak negatives after an initial warm-up period to handle incomplete annotations common in retrospective datasets.

Key Contributions

• Architecture: JANUS operationalizes the principle that quantitative measurements should constrain visual predictions, not just augment them. It introduces a multiplicative bottleneck that structurally enables the suppression of visual evidence when it conflicts with macro-radiomic priors.
• Metric (Physiological Veto Rate - PVR): The authors propose PVR to quantify the suppression of high-confidence baseline false positives. They also define Veto Selectivity (ratio of false-positive suppression to true-positive suppression) to demonstrate that the gating mechanism is selective rather than uniformly reducing confidence.
• Empirical Validation: The study provides a rigorous comparison against baselines, including a standard ViT and ORACLE-CT variants, demonstrating that the specific gating mechanism outperforms additive fusion strategies.

Results

• Performance: On the MERLIN test set ($N=5,082$), JANUS achieved a macro-AUROC of 0.88 and AUPRC of 0.74, outperforming all reproduced baselines (ViT-Baseline: 0.84; ORACLE-CT: 0.86).
• Generalization: On an external dataset ($N=2,000$) from a different institution, JANUS maintained strong performance (AUROC 0.87), significantly outperforming baselines which showed performance drops or stagnation.
• Calibration: JANUS demonstrated superior calibration (lower Expected Calibration Error, ECE) on both internal and external datasets.
• Selectivity: The Physiological Veto Rate analysis showed that under domain shift, JANUS reduced high-confidence false positives by 30.8% while affecting only 2.85% of true positives, resulting in a veto selectivity of 10.8$\times$.
• Robustness: Under simulated scalar corruption (up to 50% noise), JANUS degraded gracefully, remaining above the ViT baseline, whereas additive fusion methods (+OSF) approached the baseline performance.
• Stratified Analysis: Gains were most pronounced for findings defined by geometry (size) and densitometry (e.g., Aortic Aneurysm, Hepatic Steatosis). For focal pathologies where organ-level priors are less informative (e.g., Appendicitis), performance remained comparable to the baseline, avoiding the degradation seen in additive fusion models.

Significance and Claims
The paper claims that JANUS demonstrates that physically grounded conditioning improves both discrimination and reliability in CT triage. By enforcing anatomical constraints through a multiplicative gating mechanism, the model becomes more robust to distribution shifts and better at suppressing spurious visual correlations that do not align with quantitative biomarkers. The authors conclude that this approach offers a clinically necessary step toward reliable automated triage, particularly for findings defined by quantitative thresholds.

Limitations
The authors acknowledge that JANUS relies on a sequential pipeline where segmentation must precede gating, which limits its utility for focal findings lacking clear organ-level proxies. Additionally, scalar robustness was evaluated via simulated corruption rather than realistic segmentation errors, and the use of CT-pretrained backbones remains future work.