Structured Spectral Graph Representation Learning for Multi-label Abnormality Analysis from 3D CT Scans

Technical Summary: Structured Spectral Graph Representation Learning for Multi-label Abnormality Analysis from 3D CT Scans

Problem Statement
The increasing volume of chest CT examinations necessitates automated tools to assist radiologists in detecting abnormalities. While multi-label classification is critical for clinical decision support, it remains challenging due to the complex spatial relationships inherent in volumetric data and the wide variability of pathologies. Existing approaches face specific limitations: 3D Convolutional Neural Networks (CNNs) often struggle to capture long-range dependencies, while Vision Transformers (ViTs) typically require extensive pre-training on large, domain-specific datasets to perform competitively. Furthermore, many existing 2.5D methods (which process slices independently or as sequences) lack explicit modeling of spatial continuity and inter-slice dependencies within the volume.

Methodology: CT-SSG
The authors propose CT-SSG (Structural Spectral Graph for Computed Tomography), a 2.5D framework that represents 3D CT volumes as structured graphs. The architecture proceeds through four main stages:

1. Feature Initialization: The input 3D volume is partitioned into $N$ non-overlapping triplets of adjacent axial slices. Each triplet is processed independently by a 2D ResNet-18 backbone to extract spatial features, resulting in a compact embedding for each triplet. This approach captures local inter-slice context without resorting to computationally expensive full 3D convolutions.
2. Graph Construction: The volume is modeled as a graph $G=(V, E)$ where each node $v_i$ represents a triplet of axial slices. Edges connect nodes based on their physical distance along the caudal-cranial (z) axis. The edge weights are defined by a function of the inter-triplet distance and slice spacing, allowing the model to encode spatial relationships. The authors explore both fully connected and sparse topologies, where connectivity is limited to a specific receptive field size $q$.
3. Spectral Domain Message Passing: To handle variability in anatomical positioning across patients, the model employs a Spectral Module based on Chebyshev graph convolutions. Unlike spatial graph convolutions that aggregate fixed local neighborhoods, spectral convolutions operate in the Fourier domain using polynomial approximations of the graph Laplacian. This allows the model to capture hierarchical feature representations and long-range anatomical relationships while remaining robust to inter-patient variability. The module includes Triplet Axial Slice Positional Embeddings to retain axial position information and utilizes residual connections with normalization layers.
4. Classification: The updated node representations are aggregated via mean pooling to produce a global volume embedding, which is passed to a classification head to predict the presence of multiple abnormalities using Binary Cross-Entropy loss.

Key Contributions
The paper outlines five primary contributions:

1. CT-SSG Framework: A novel visual encoder modeling 3D CT volumes as graphs of triplet axial slices, incorporating Triplet Axial Slice Positional Embeddings and edge-weighting strategies within a spectral-domain GNN module.
2. Cross-Dataset Generalization: Demonstrated strong generalization when trained on the CT-RATE dataset (Turkey) and evaluated on independent datasets from the US (RAD-ChestCT) and France (CT-HCL). The model also showed transferability from chest to abdominal CT scans.
3. Comprehensive Ablation Studies: Systematic evaluation of model depth, hyperparameters, graph topology (sparse vs. fully connected), edge weighting, and connectivity patterns. The study identifies that sparse topologies with local neighborhoods and a single spectral layer yield optimal performance.
4. Transfer to Report Generation: Evaluation of CT-SSG as a visual encoder for automated radiology report generation, demonstrating that the learned representations are effective for downstream text generation tasks.
5. Transfer to Abdominal CT: Validation of the approach in a low-data regime for abdominal CT abnormality classification, showing that chest-pretrained backbones outperform supervised training from scratch when fewer than 3,750 samples are available.

Experimental Results
The model was evaluated on the CT-RATE dataset (21,304 patients) and compared against 3D CNNs, 3D Transformers (ViT3D, ViViT, Swin3D), and other 2.5D methods (CT-Net, CT-Scroll).

• Performance: CT-SSG achieved a macro F1-Score of 57.18% on the CT-RATE test set, outperforming the best baseline (CT-Scroll, 54.30%) and showing significant gains over 3D Transformers and CNNs.
• Generalization: In cross-dataset evaluations, CT-SSG consistently ranked highest in F1-Score, AUROC, and mAP on both RAD-ChestCT and CT-HCL, indicating robustness to different clinical distributions.
• Ablation Findings:
  • Topology: Sparse graph topologies (receptive field $q=16$) outperformed fully connected topologies, suggesting that restricting aggregation to local interactions better captures short-range dependencies.
  • Depth: A single spectral layer yielded the best results, implying that deep graph structures are not necessary for this task.
  • Filter Size: In sparse graphs, a Chebyshev filter size of $K=3$ was optimal, whereas fully connected graphs showed diminishing returns with higher $K$.
  • Robustness: The model remained robust to z-axis translations and Gaussian noise perturbations.
• Downstream Tasks: In report generation, CT-SSG achieved a 38.72% F1-Score (clinical efficacy), significantly outperforming ViViT (27.54%) and ResNet3D (25.45%). In abdominal CT transfer, linear probing on frozen chest-pretrained weights outperformed supervised training from scratch in low-data regimes.

Significance and Claims
The authors claim that CT-SSG provides a flexible and general paradigm that bridges 2D and volumetric modeling. By representing CT volumes as structured graphs, the method explicitly controls neighborhood definitions and edge weighting, offering a physically grounded alternative to the dense connectivity of Transformers. The work suggests that explicit structural priors (such as local inter-slice connectivity) serve as an effective inductive bias for 3D reasoning, achieving competitive performance with lower computational complexity compared to 3D Transformers. The authors position CT-SSG as a versatile backbone suitable for a broad range of 3D medical imaging tasks, including abnormality detection and report generation, without requiring extensive domain-specific pre-training.