Technical Summary: ARMOR (Antibiotic Resistance Modelling through Omics and Resistance-gene analysis)

Problem Statement

Antimicrobial resistance (AMR) in Klebsiella pneumoniae is a critical global health threat, responsible for over 214,000 attributable deaths annually. Clinically, the standard culture and sensitivity testing required to determine resistance profiles takes 48–72 hours, forcing clinicians to prescribe antibiotics empirically during a window where treatment failure and resistance spread are likely. While machine learning (ML) approaches using whole-genome sequencing (WGS) offer a faster alternative, existing tools face three specific gaps:

1. Limited Scope: No published ML model targets amikacin resistance in K. pneumoniae as a primary endpoint, despite it being a critical aminoglycoside.
2. Exclusion of Key Antibiotics: High-quality frameworks like PanKA exclude amikacin and fosfomycin from their analyses.
3. Deployment Barriers: Existing tools are typically Python command-line scripts incompatible with hospital Laboratory Information Systems (LIS), lacking a compiled, hospital-compatible deployment pathway.

Furthermore, early AMR prediction systems often rely on raw DNA k-mers, which represent a sparse signal (resistance genes constitute <2% of the genome), forcing classifiers to detect signatures buried in genomic noise.

Methodology

ARMOR addresses these gaps through a specialized machine learning pipeline designed to extract biologically meaningful features rather than relying on raw genomic noise.

Data Curation

• Source: Genomic and phenotypic data were sourced from the Bacterial and Viral Bioinformatics Resource Center (BV-BRC).
• Filtering: From an initial 123,181 records, the authors filtered for K. pneumoniae (Taxon ID: 573) with experimental laboratory method evidence only, excluding computationally predicted labels to avoid circular reasoning.
• Final Dataset: 8,733 records across 3,745 unique genomes were retained. Intermediate phenotypes were excluded to enforce binary classification (Resistant vs. Susceptible).
• Target Antibiotics: Amikacin, Cefepime, Piperacillin/Tazobactam, and Fosfomycin.

Feature Engineering

ARMOR constructs four categories of biologically specific features to increase signal density to an estimated 65–75%:

1. Pangenome Presence/Absence: A matrix of ~32,000 accessory gene families derived from a Panaroo-constructed pangenome (2,507 genomes).
2. AMR Gene Features: Binary features for 742 resistance gene families called via RGI/CARD at a 90% identity threshold.
3. Protein K-mers: Frequency vectors (~7,000 features) from detected resistance genes to capture variant-level differences (e.g., CTX-M-15 vs. CTX-M-27).
4. Core Gene SNPs: Six curated variants in genes associated with known resistance mechanisms (e.g., gyrA, parC, murA, porins).

Novelty in Fosfomycin Modeling:
Unlike other antibiotics, fosfomycin resistance is modeled via an explicit multi-mechanism feature set. This integrates four independent mechanisms (FosA plasmid variants, MurA mutations, GlpT/UhpT transporter loss) and three epistatic interaction features. This approach captures co-occurring pathways, transforming fragmented signals into a compact, interpretable representation.

Feature Curation:
The authors employed allele-aware prevalence thresholds to exclude near-invariant features (e.g., intrinsic fosA5/fosA6 alleles in K. pneumoniae) and artifacts (e.g., homology artifacts in uhpT calls), retaining only discriminative acquired determinants.

Training Strategy

• Algorithm: LightGBM binary classifiers trained independently for each antibiotic (Binary Relevance strategy) to avoid spurious correlations between mechanistically unrelated resistance genes.
• Validation:
  • Internal: 5-fold stratified cross-validation (10-fold for fosfomycin due to smaller sample size).
  • External (Within BV-BRC): A strict BioProject ID-based hold-out strategy. Genomes from specific BioProjects were entirely withheld from training to prevent strain-level leakage, simulating external validation within the same database.
• Handling Imbalance: Class imbalance was addressed via the WeightOfPositiveExamples parameter. SMOTE was applied only within training folds for antibiotics with low resistance prevalence (e.g., amikacin), never to validation data.
• Thresholding: For external validation of amikacin, where resistance prevalence was extremely low (4.8%), the standard 0.5 threshold collapsed sensitivity. The authors utilized the Youden's J-optimal operating point (threshold = 0.0165) to reflect discriminative performance.

Key Results

Performance Metrics (BioProject Hold-Out Evaluation)

ARMOR achieved state-of-the-art or competitive performance across the four targets:

• Amikacin: AUC-ROC 0.9864, F1-Score 0.8809. This represents the highest published AUC for amikacin in K. pneumoniae.
• Piperacillin/Tazobactam: AUC-ROC 0.9394, F1-Score 0.8680.
• Cefepime: AUC-ROC 0.9075, F1-Score 0.8766.
• Fosfomycin: AUC-ROC 0.8158 (10-fold CV), outperforming previous rule-based methods (0.57 accuracy) and the Sevilla-Fortuny et al. (2024) model (0.78).

Comparison with Existing Tools

ARMOR outperformed or matched existing models (KSSHIBA, Sevilla-Fortuny et al., PanKA) across all reported metrics. Notably, for amikacin and piperacillin/tazobactam, ARMOR achieved the highest published AUCs.

Generalizability and External Validation

A critical finding was the divergence in generalizability when tested on an independent external cohort (n≈145 per antibiotic):

• Amikacin: Retained strong discriminative power (AUC 0.836), though probability calibration required adjustment (threshold shift) due to the lower resistance prevalence in the external set.
• Beta-Lactams (Cefepime & Pip/Tazo): Performance collapsed to near-chance levels (AUC ~0.57).
• Interpretation: SHAP analysis revealed that beta-lactam models relied heavily on pangenome accessory-gene features acting as lineage co-occurrence markers rather than direct mechanistic drivers. These lineage-specific signatures failed to transfer to the genomically distinct external population. In contrast, amikacin predictions were driven by direct CARD allele-level enzyme calls, which proved more portable.

Interpretability (SHAP)

SHAP analysis confirmed that the model utilizes biologically plausible drivers:

• Cefepime: ompK36 disruption (porin loss) and beta-lactamase genes were top drivers.
• Amikacin: Accessory membrane protein clusters (e.g., LamB) and ABC transporters were top features, likely reflecting lineage associations with aminoglycoside-modifying enzymes.
• Fosfomycin: The model correctly prioritized fosA3 (acquired) and glpT disruption, validating the multi-mechanism feature design.

Significance and Claims

The paper claims that ARMOR demonstrates biologically informed feature engineering substantially outperforms raw DNA k-mer approaches for AMR prediction.

1. Methodological Contribution: The study establishes that constructing features based on pangenome presence/absence, protein k-mers, and mechanism-specific interactions (especially for multi-mechanism drugs like fosfomycin) yields higher signal density and accuracy than raw sequence data.
2. Gap Filling: It provides the first published ML model for amikacin resistance in K. pneumoniae and includes fosfomycin, addressing specific blind spots in current literature.
3. Transparency and Generalizability: Through rigorous SHAP analysis and BioProject-level validation, the paper explicitly identifies why generalizability fails for certain antibiotics (reliance on lineage markers vs. direct mechanisms). This diagnostic capability is presented as practically important for genomic surveillance and future clinical translation.
4. Reproducibility: All code, trained models, and processed datasets are openly released, and the pipeline is designed to be extensible to other pathogens, though the current scope is limited to K. pneumoniae.

The authors conclude that while ARMOR achieves high accuracy within the BV-BRC ecosystem, real-world deployment requires site-specific threshold recalibration and further external validation, particularly for beta-lactam antibiotics where lineage-dependent features limit cross-population performance.