🤖 AI

When is 3D Worth It? A Resource-Performance Frontier for CNNs and Transformers in Lung CT

This study establishes a resource-performance frontier for lung CT classification, demonstrating that 2.5D CNNs offer a more reliable trade-off between performance, stability, and computational efficiency than 3D models or Vision Transformers, which suffer from threshold instability and degenerate predictions.

Original authors: Md Enamul Hoq, Sharafat Hossain, Imraul Emmaka, Linda Larson-Prior, Lawrence Tarbox, Jonathan Bona, Donald Johann Jr. and Fred Prior

Published 2026-06-08

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Md Enamul Hoq, Sharafat Hossain, Imraul Emmaka, Linda Larson-Prior, Lawrence Tarbox, Jonathan Bona, Donald Johann Jr. and Fred Prior

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to find a tiny, hidden needle in a massive, complex haystack. In the world of medical imaging, that "haystack" is a 3D CT scan of a human lung, and the "needle" is a small sign of cancer.

For a long time, doctors and computer scientists assumed that to find the needle best, you needed to look at the entire haystack at once. This is the "3D" approach: feeding the computer the full, volumetric block of the lung. The paper you provided asks a simple but crucial question: "Is looking at the whole haystack actually worth the extra effort, or is there a smarter, cheaper way?"

Here is the breakdown of their experiment and what they found, using everyday analogies.

The Setup: Three Ways to Look at the Haystack

The researchers tested three different ways to show the lung scan to the computer, using two different types of "brain" models (one based on traditional patterns called CNNs, and one based on newer, attention-based patterns called Transformers):

2D (The Single Snapshot): They showed the computer just one flat slice of the lung, like looking at a single page of a book.
2.5D (The Three-Page Spread): They showed three slices stacked together (front, side, and top views), like opening a book to three pages at once to get a bit more context.
3D (The Whole Book): They fed the computer the entire volume of the lung, like trying to read the whole book at once in your mind.

They kept everything else exactly the same: the same data, the same training rules, and the same goal. They just changed how the "picture" was presented.

The Results: The "Goldilocks" Winner

The study found that bigger isn't always better. In fact, the biggest approach often broke the computer's brain.

The 3D Approach (The "Overwhelmed" Student):
When they tried to feed the full 3D volume to the models, things went wrong.
- The 3D CNN became "unstable." It was like a student who knows the material but gets so nervous during the test that they can't decide whether to answer "Yes" or "No." Sometimes it said yes, sometimes no, with no consistency.
- The 3D Transformers (the newer, more complex models) completely "collapsed." They acted like a broken alarm system that just screamed "ALARM!" for every single patient, regardless of whether they were sick or healthy. They gave up on nuance and just guessed "Yes" for everyone.
The 2D Approach (The "Too Cautious" Student):
The single-slice model was too scared to make a mistake. It was like a security guard who refuses to let anyone in because they are afraid of letting a thief through. It was very accurate at saying "No" to healthy people, but it missed almost all the sick people (it was too conservative).
The 2.5D Approach (The "Sweet Spot"):
The 2.5D CNN was the clear winner. It was the "Goldilocks" solution.
- It gave the computer just enough context (three slices) to understand the shape of the lung without overwhelming it with data.
- It was stable: It didn't flip-flop between answers.
- It was efficient: It didn't require massive amounts of computer power (memory) to run.
- It offered the best balance: It found the "needles" (cancer) reasonably well without getting confused or crashing.

The Catch: "Worth It" Depends on the Cost

The authors are very honest about the limitations. They didn't find a magic bullet that cures cancer.

The Confidence Interval: The results were a bit "fuzzy." The difference between the best model and the others wasn't statistically huge, mostly because the dataset was small (only about 20 sick patients in the test group).
The Failure Modes: The most important discovery wasn't just about who won, but how the others lost. The 3D models didn't just perform slightly worse; they exhibited "degenerate" behavior (like predicting "all positive" for everyone). This means that simply adding more data (3D) didn't make the models smarter; it actually made them more prone to giving up and guessing randomly.

The Bottom Line

The paper concludes that for the specific task of screening (looking through many scans to find potential cancer), you don't need the heavy, expensive, full-3D machinery.

Think of it like this: If you are looking for a typo in a single sentence, you don't need to read the entire encyclopedia. A focused look at a few lines (2.5D) is often faster, cheaper, and less likely to confuse you than trying to hold the whole book in your head at once.

In short: For lung cancer screening, the "middle ground" (2.5D) is the most practical and reliable choice, while the "full 3D" approach is currently too expensive and unstable to be worth the extra cost.

Technical Summary: When is 3D Worth It? A Resource–Performance Frontier for CNNs and Transformers in Lung CT

Problem Statement
While three-dimensional (3D) models are widely assumed to be superior for volumetric medical imaging due to their ability to capture volumetric context, their practical utility is often questioned regarding whether performance gains justify the added computational cost, memory requirements, and optimization complexity. Despite the prevalence of 3D approaches in lung CT analysis, there is a lack of controlled evidence determining when 3D inputs are actually worth the cost compared to 2D or hybrid 2.5D strategies, particularly within class-imbalanced screening scenarios.

Methodology
The authors conducted a controlled, diagnostic study rather than proposing a new architecture. The study utilized a leakage-free National Lung Screening Trial (NLST) cohort ( $n=1,977$ ) with fixed patient-level splits (1,426 train, 254 validation, 297 test). The task was binary patient-level cancer classification, characterized by strong class imbalance (approx. 6.7% prevalence in the test set). A supporting cohort from LIDC-IDRI was used solely for external corroboration of failure modes, not for training.

The experimental design isolated input dimensionality as the primary variable while holding the training protocol constant:

Inputs: Three input types were constructed from a single canonical CT series per patient:
- 2D: A single central axial slice.
- 2.5D: Three orthogonal slices stacked as channels.
- 3D: A lung-centered sub-volume.
Models: Two backbone families were adapted to each input dimension: a Residual Convolutional Neural Network (CNN) and a Vision Transformer (ViT).
Protocol: All models were trained for 20 epochs using weighted binary cross-entropy to address imbalance, with identical optimizer and learning-rate settings.
Evaluation: Performance was assessed via ROC-AUC, PR-AUC, sensitivity, and specificity at both default (0.5) and validation-optimized thresholds. Computational cost was measured via GPU memory and inference time. Bootstrap 95% confidence intervals were calculated for AUC metrics.

Key Results
The study found that increased dimensionality did not guarantee stable or superior performance in this imbalanced setting:

2.5D CNN: Achieved the most favorable trade-off, yielding the highest ROC-AUC (0.682, 95% CI [0.546, 0.799]) and PR-AUC (0.158) with a stable operating point and low GPU memory usage (~1.6 GB).
3D CNN: Exhibited significant threshold instability, with sensitivity varying drastically (0.05 to 0.50) across different thresholds, despite a moderate ROC-AUC (0.622).
Transformers (ViT): Suffered from degenerate predictions. The 3D ViT produced zero sensitivity (all-negative predictions) despite an ROC-AUC of 0.589, while the 2.5D ViT showed extreme threshold sensitivity. Transformers generally required roughly 3× the GPU memory of CNNs.
Statistical Significance: The authors emphasize that bootstrap confidence intervals were wide and overlapping across all models. Consequently, the ranking differences in point estimates are not statistically separable.

Key Contributions

Resource–Performance Frontier: The paper maps the trade-off between input dimensionality, computational cost, and discrimination for both CNN and ViT families under a fixed protocol.
Failure-Mode Taxonomy: The study characterizes specific operating-point failures that ROC-AUC alone conceals, specifically "threshold instability" (in 3D CNNs) and "degenerate predictions" (e.g., all-positive or all-negative outputs in Transformers).
Diagnostic Scope: Rather than claiming a new state-of-the-art model, the work serves as a controlled, reproducible starting point for larger studies, explicitly treating the single-cohort scope and modest absolute performance as limitations.

Significance and Claims
The paper modestly claims that for class-imbalanced lung cancer screening classification, 2.5D inputs provide a more reliable trade-off between performance, stability, and computational efficiency than full 3D representations. The authors argue that while 3D context is theoretically beneficial, it does not guarantee stable behavior in limited-data, imbalanced regimes and may introduce optimization difficulties, particularly for data-hungry architectures like Transformers.

The findings suggest that 2.5D should be considered a practical default and a starting hypothesis for lung CT classification tasks, whereas full 3D representations may be more appropriate for spatially intensive downstream tasks (e.g., segmentation) where volumetric continuity is critical. The authors conclude that the suitability of input dimensionality is task-dependent and that the observed resource–performance frontier and failure modes are observations to be confirmed in larger, multi-cohort studies rather than definitive superiority claims.

The Setup: Three Ways to Look at the Haystack

The Results: The "Goldilocks" Winner

The Catch: "Worth It" Depends on the Cost

The Bottom Line

More like this