📄 medicine

A Smoking-Independent CT Radiomic Signal Predicts EGFR Mutation in Non-Small-Cell Lung Cancer Under a Pre-Registered, Cross-Validated Design

Under a rigorous, pre-registered, and cross-validated design that explicitly controls for smoking confounding, this study demonstrates the existence of a modest, smoking-independent CT radiomic signal for predicting EGFR mutation in NSCLC (AUC ~0.63), while revealing that previously reported higher accuracies likely stem from uncorrected confounding and methodological optimism.

Original authors: Adil Karim, Amine Berquedich, El Habib Benlahmar, Sanaa El Filali, Ahmed Zellou

Published 2026-08-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Adil Karim, Amine Berquedich, El Habib Benlahmar, Sanaa El Filali, Ahmed Zellou

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the fight against non-small-cell lung cancer, doctors rely on two critical pieces of information to decide how to treat a patient. First, they need to know if the tumor carries a specific genetic change called an EGFR mutation, which makes the cancer vulnerable to a certain class of targeted drugs. Second, they need to measure the level of a protein called PD-L1, which helps determine if immunotherapy will work. Traditionally, obtaining this information requires a biopsy, a procedure where a needle extracts a piece of the tumor for laboratory analysis. This process is invasive, takes time, and sometimes yields only a small sample. Scientists have long hoped to bypass the needle entirely by reading these genetic secrets directly from a routine CT scan, a standard X-ray image of the chest. The idea is that the texture and patterns of the tumor on the image might act as a fingerprint for its genetic makeup, a field of study known as radiogenomics. However, while many studies have claimed high success rates in this area, the results have been inconsistent, and critics have worried that the models were not actually reading the tumor's biology but were instead picking up on a simpler, unrelated clue.

A team of researchers set out to settle this uncertainty with a study designed to be as strict and honest as possible. They asked a simple, conservative question: can a CT scan predict an EGFR mutation without simply detecting whether the patient is a smoker? This distinction is vital because EGFR mutations are far more common in people who have never smoked, while KRAS mutations are more common in smokers. If a computer model learns to predict the mutation by looking for signs of smoking in the image, it is not truly reading the tumor; it is just reading the patient's history. The researchers analyzed data from 169 patients, using a pre-registered plan that locked in their methods before they saw the results to prevent them from accidentally tweaking the process to get a better score. They extracted 94 different measurements from the CT images, describing the tumor's shape and texture, and fed them into a machine learning model.

The results revealed a genuine, albeit modest, signal. The model could predict the EGFR mutation with an accuracy that was statistically better than random guessing, achieving a score of 0.633. More importantly, the researchers proved this signal was not a trick of the smoking history. When they tested the model specifically on patients who had never smoked, the accuracy remained strong, and the model's predictions actually moved in the opposite direction of smoking status, confirming it was looking at the tumor itself rather than the patient's lifestyle. The team also tested the model's ability to predict KRAS mutations and PD-L1 levels. For KRAS, the model performed no better than a coin flip, effectively proving that the system was not hallucinating results. For PD-L1, the results were weak and barely reached the threshold of significance, suggesting that this particular protein is much harder to detect from an image than the EGFR mutation.

Perhaps the most revealing part of the study was what happened when the researchers tried to make the model look better by using common shortcuts. In many previous studies, researchers might test dozens of different models and report only the one that worked best, or they might let the model learn from the entire dataset before testing it, which creates an illusion of success. When this team applied those same shortcuts to their data, the accuracy jumped to 0.81, matching the high numbers seen in other published papers. This demonstrated that the inflated numbers in the literature were likely the result of these methodological shortcuts and the confounding effect of smoking, rather than a true breakthrough in reading biology from images. The researchers also tried using a complex deep learning system, a type of artificial intelligence that learns directly from the raw pixels of the image, but it failed to outperform the simpler, traditional method.

The study concludes that a real, smoking-independent signal for EGFR mutations does exist in CT scans, but it is smaller and more difficult to find than the field has previously claimed. The signal is strongest in the group of patients who need it most: those who have never smoked and for whom smoking-based guesses are useless. However, the researchers are careful to note that this is not yet a ready-to-use medical test. The accuracy is not high enough to replace a biopsy, and the finding needs to be confirmed on a completely different set of patients from another hospital to ensure it works in the real world. By stripping away the optimism and the confounding variables, the study provides a clearer, more honest map of what is actually possible, showing that while the dream of reading genetics from a scan is not a fantasy, it is also not as close to reality as some headlines suggest.

Technical Summary: A Smoking-Independent CT Radiomic Signal Predicts EGFR Mutation in NSCLC

Problem Statement
Computed tomography (CT) radiogenomic models have reported high accuracies (pooled AUC ~0.80) for predicting epidermal growth factor receptor (EGFR) mutation status in non-small-cell lung cancer (NSCLC). However, these results are frequently undermined by three methodological flaws: (1) failure to control for the strong confounding association between smoking history and EGFR status (EGFR mutations are enriched in never-smokers); (2) optimistic evaluation practices, such as feature selection outside of cross-validation and lack of multiplicity correction; and (3) the absence of a rigorous baseline to determine if imaging adds value over clinical variables. The field lacks a definitive answer to whether a genuine, smoking-independent radiomic signal for EGFR exists when subjected to strict statistical controls.

Methodology
The authors employed a deliberately conservative, pre-registered, and cross-validated design using the publicly available NSCLC-Radiogenomics cohort (TCIA).

Cohort: The study analyzed 169 patients for EGFR mutation status (43 positive), 168 for KRAS mutation (35 positive, serving as a pre-registered negative control), and 111 for CD274 (PD-L1) gene expression.
Features: 94 standardized radiomic features were extracted following the Image Biomarker Standardization Initiative (IBSI), including first-order statistics and texture matrices (GLCM, GLRLM, GLSZM, GLDM, NGTDM).
Primary Model: A Random Forest (RF) classifier was fixed a priori as the primary model to capture non-linear interactions with low variance.
Validation Protocol:
- Cross-Validation: Five-fold stratified cross-validation, repeated over ten random seeds to ensure stability.
- Permutation Testing: Significance was assessed via label shuffling (400 permutations for EGFR) to generate null distributions.
- Multiplicity Correction: Bonferroni correction was applied across the three biomarker targets.
- Confound Control: A pre-registered "smoking-confound gate" included: (i) testing performance in the never-smoker stratum; (ii) checking correlation between predictions and smoking status; and (iii) residualizing all features against a smoking design matrix to ensure orthogonality.
Comparative Analyses:
- Benchmarking: 18 different model classes (including deep learning, SVMs, and boosting ensembles) were benchmarked.
- Improvement Attempts: A battery of nested cross-validation strategies (recursive feature elimination, hyperparameter tuning, stacking, peritumoral/habitat features) was tested to see if the performance ceiling could be raised.
- Deep Learning: An end-to-end 3D Convolutional Neural Network (CNN) was trained on CT patches to compare against hand-crafted radiomics.
- Reconstruction of Literature: The authors simulated "optimistic" pipelines (e.g., feature selection on the full dataset, reporting best splits, adding clinical confounds) to reproduce the field's reported ~0.80 AUC.

Key Results

EGFR Prediction: The pre-specified Random Forest achieved an out-of-fold (OOF) AUC of 0.633 (mean over 10 seeds). This result survived permutation testing ( $p=0.010$ ) and Bonferroni correction ( $p_{adj}=0.030$ ).
Smoking Independence: The signal was confirmed as smoking-independent:
- It was anti-correlated with a smoking predictor ( $\rho = -0.28$ ).
- It persisted after removing the smoking component from all features (AUC 0.708).
- It held in the never-smoker subgroup (AUC 0.70, 95% CI 0.53–0.85), where smoking history provides no information.
Negative Controls:
- KRAS: Prediction was no better than random guessing (AUC 0.498, $p=0.48$ ), confirming the pipeline does not manufacture false signals.
- PD-L1/CD274: Showed only a borderline trend (Spearman $\rho=0.213$ , $p=0.0499$ ) which failed Bonferroni correction.
Model Benchmarking: While tree-based ensembles (AdaBoost, Gradient Boosting) generally outperformed linear and neural models, the "best" model found via search failed significance after correcting for the "best-of-eighteen" search. The pre-specified RF remained the only statistically defensible positive.
Deep Learning: The 3D-CNN performed at chance level (AUC 0.523), failing to outperform hand-crafted radiomics or the clinical baseline.
Performance Ceiling: No improvement strategy (tuning, stacking, adding peritumoral features) significantly exceeded the baseline RF. The attainable ceiling for this cohort is ~0.63.
Reconstruction of Optimism: By combining common evaluation shortcuts (full-dataset feature selection, best-split reporting) and adding clinical variables (the smoking confound), the authors reproduced an AUC of 0.807 on the same data, matching the literature's pooled average. This demonstrates that headline figures of ~0.80 are recoverable from optimism and confounding rather than transferable signal.

Significance and Claims
The paper claims to establish the existence of a modest, genuine, and smoking-independent radiomic signal for EGFR mutation in NSCLC.

Modesty of Claim: The authors explicitly state they do not claim a superior AUC to the literature's 0.80. Instead, they report a lower number (0.63) that has "paid the costs" other studies defer (pre-registration, confound control, multiplicity correction).
Clinical Relevance: The signal is strongest in never-smokers, the subgroup where clinical heuristics based on smoking history fail and EGFR testing is most critical.
Methodological Contribution: The study serves as a rigorous benchmark, demonstrating that "the best of many models" is a statistical illusion and that simple tree-based models are sufficient at this sample size. It highlights that the gap between development-set performance and independent validation in the field is largely due to uncontrolled confounding and optimistic evaluation.
Future Direction: The authors conclude that while a signal exists within this single cohort, confirmation on a separate cohort from another center is the necessary next step to establish clinical utility and transferability. They frame their result as evidence of a genuine imaging signal, not as a deployable diagnostic test.

Technical Summary: A Smoking-Independent CT Radiomic Signal Predicts EGFR Mutation in NSCLC

More like this