Brain as Language: Neuro-Semantic Modeling of Glioblastoma Radiogenomics

Technical Summary: Brain as Language: Neuro-Semantic Modeling of Glioblastoma Radiogenomics

Problem Statement
Glioblastoma (GBM) is characterized by pronounced spatial, vascular, and molecular heterogeneity, which limits the inferential power of single-tissue biopsies. While multiparametric MRI (mpMRI) captures this heterogeneity, conventional radiogenomic pipelines typically encode imaging data as high-dimensional numerical feature matrices. This approach creates a "representation problem": while quantitative detail is preserved, the semantic structure through which clinicians reason (e.g., enhancement heterogeneity, infiltration, necrotic burden) is lost. Models operate on abstract feature names (e.g., entropy, kurtosis) rather than clinically interpretable phenotype concepts, hindering explainable AI (XAI) and evidence-grounded reasoning. The study addresses the need for an intermediate representation layer that translates quantitative radiomic evidence into controlled, interpretable, and auditable biomedical language.

Methodology
The study utilizes the University of Pennsylvania Glioblastoma (UPenn-GBM) dataset, comprising preoperative mpMRI, expert segmentations, and molecular markers. The core methodology involves constructing a "neuro-semantic radiogenomic language" through the following steps:

1. Data Structure: Radiomic features were extracted from 11 imaging sources (including T1, T1CE, T2, FLAIR, DTI, and DSC maps) across three tumor subregions: edema/infiltrative (ED), enhancing tumor (ET), and necrotic core (NC).
2. Ontology Construction: 86 raw radiomic features were mapped to a controlled ontology. Features were grouped into three base semantic domains (texture complexity, intensity distribution, morphology) and refined by modality context (e.g., texture in DSC maps became "perfusion heterogeneity"; texture in FLAIR became "infiltration-associated heterogeneity").
3. Tokenization: Patient-specific radiomic values were converted into cohort-relative percentile levels (LOW, MODERATE, HIGH). These were encoded into structured tokens following the grammar: MODALITY_REGION_LEVEL_SEMANTIC_DOMAIN. For example: DSC_APRCBV_ED_MODERATE_PERFUSION_HETEROGENEITY.
4. Sequence Generation: Patient profiles were assembled into "radiogenomic sentences" (up to 99 tokens) bounded by start/end markers.
5. Modeling Strategy:
   • MGMT Prediction: The framework was evaluated for predicting O6-methylguanine-DNA methyltransferase (MGMT) promoter methylation status (n=278 definitive cases) using repeated stratified nested 5-fold cross-validation.
   • Representations Tested: (a) Coarse semantic tokens with TF-IDF and logistic regression; (b) Binary token encoding with nonlinear classifiers (XGBoost, SVM, RF); (c) Hierarchical semantic tokens preserving feature-specific descriptors; (d) Hybrid models combining hierarchical tokens with aggregated quantitative radiomic evidence.
   • LLM Reasoning: Three Large Language Models (BioMistral-7B, Meditron-70B, and a general instruction-tuned model) were evaluated as explanation layers. They received the radiogenomic sentence, the model's MGMT prediction, and the token evidence table to generate explanations. Performance was measured by claim support rate, hallucination rate, and expert usefulness.

Key Results

• Representation Feasibility: The pipeline successfully generated 652 radiogenomic sequences from the UPenn-GBM cohort, with 611 linked to clinical metadata. The median token count was 99, indicating high coverage of the multi-view imaging structure.
• Predictive Performance (MGMT):
  • A coarse semantic representation with linear TF-IDF achieved a low ROC AUC of 0.55, indicating insufficient discrimination.
  • Introducing binary token encoding and nonlinear classifiers (XGBoost) improved the ROC AUC to 0.76.
  • Hierarchical Semantic Modeling: Retaining feature-specific descriptors within semantic domains increased the ROC AUC to 0.80 (95% CI, 0.75–0.85).
  • Hybrid Radiomic-Semantic Modeling: Combining hierarchical tokens with residual quantitative evidence yielded the highest performance, with a ROC AUC of 0.83 (95% CI, 0.78–0.88), an accuracy of 0.78, and an F1-score of 0.76.
• LLM Explanation Fidelity:
  • Biomedical LLMs (BioMistral and Meditron) significantly outperformed the general instruction-tuned model in evidence fidelity.
  • BioMistral achieved a claim support rate of 0.94 and a contradiction rate of 0.01, compared to 0.78 and 0.06 for the general model.
  • The structured token input acted as an "anti-hallucination scaffold," ensuring that generated explanations were grounded in specific patient evidence rather than fluent but unsupported biomedical text.

Significance and Claims
The authors claim that the primary contribution of this work is a structured representation layer that makes radiomic evidence interpretable, modelable, and auditable. The study reframes radiogenomics not merely as a classification task but as a representation problem.

Key claims include:

1. Interpretability-Information Trade-off: Coarse semantic tokens are readable but compress too much information. Hierarchical tokens and hybrid models successfully balance clinical interpretability with the preservation of molecularly relevant signal.
2. Hybrid Superiority: The most effective approach is not purely numerical or purely semantic, but a structurally integrated hybrid that combines semantic abstraction with residual quantitative detail.
3. Auditable Reasoning: The neuro-semantic language enables a new paradigm for biomedical LLMs where explanations are not autonomous predictions but conditional generations constrained by verifiable evidence. This allows for claim-level auditing of AI reasoning in clinical contexts.

The authors maintain modesty regarding their findings, noting that the results are based on a single public cohort (UPenn-GBM) and require confirmation in independent datasets. They position the framework as a proof-of-concept for a structured basis for interpretable model evaluation and future integration with trustworthy biomedical language models.