Technical Summary: Predictive Value of Structured Digital Features vs. Clinical Text for Liver Failure

Problem Statement

The prediction of inpatient liver failure, a condition with high mortality (50–90%) and a diagnosis inherently anchored to objective numerical thresholds (e.g., INR, bilirubin, creatinine), has been approached through two contrasting machine learning paradigms: structured digital features (laboratory values, vitals) and unstructured clinical text (radiology reports, discharge summaries). While domain-adapted language models (e.g., ClinicalBERT) have shown promise in general clinical NLP, the relative predictive value of these two modalities for liver failure has not been systematically quantified. Prior studies often suffer from confounding due to cross-cohort comparisons, making it unclear whether clinical text provides independent predictive value beyond what structured data already captures, or if text represents an informational redundancy for numerically defined conditions.

Methodology

Data Source and Cohort Construction

The study utilized a unified cohort of 89,925 inpatient admissions from MIMIC-IV version 3.1 (Beth Israel Deaconess Medical Center, 2008–2019).

• Inclusion Criteria: Admission duration ≥24 hours, at least one liver-related radiology report, and at least one target laboratory measurement.
• Outcome Label: A rigorous three-layer composite definition was used to define positive liver failure cases:
  1. ICD Codes: Discharge diagnoses containing specific ICD-9/10 codes for liver failure.
  2. Laboratory Criteria: Peak INR > 1.5 AND peak total bilirubin > 2.0 mg/dL (aligned with AASLD criteria).
  3. Clinical Outcome: In-hospital mortality.
  • Result: 17,788 positive cases (19.8% positive rate).
• Data Partitioning: Patient-level stratified random splitting ensured identical training (70%), validation (10%), and test (20%) sets across all models, eliminating cross-cohort confounding. The final test set comprised 17,916 patients.

Feature Engineering

• Structured Digital Features: 20 laboratory indicators (e.g., ALT, AST, bilirubin, INR, creatinine) were extracted.
  • Static Models: Statistical summaries (mean, max, min, last value) over the first 24 hours, augmented with 4 clinical features (age, sex, emergency flag, liver history), resulting in an 84-dimensional vector.
  • Temporal Models: Lab records were aligned to 6-hour bins over a 72-hour window, creating a tensor of shape [N, 12, 20]. Missing values were handled via forward-filling followed by median imputation.
• Text Features: Three configurations were tested using Bio_ClinicalBERT:
  • T1: Radiology reports only (Findings/Impression sections).
  • T3: Discharge summaries only.
  • T2: Concatenated radiology and discharge texts.
  • Text was truncated to 2,000 characters (or 1,000 each for concatenation).

Model Architectures

The study evaluated four experimental arms on the identical test set:

1. Static Digital Models: XGBoost and Multi-Layer Perceptron (MLP) using 24-hour snapshots.
2. Temporal Digital Models: BiLSTM and a custom Clinical Transformer (4-layer encoder) evaluated at four prediction windows (6h, 12h, 24h, 48h).
3. Language Models: Fine-tuned Bio_ClinicalBERT on the three text configurations (T1, T2, T3).
4. Multimodal Late Fusion: Equal-weight averaging of probabilities from digital and language models to assess incremental value.

Evaluation Metrics

Primary metrics included AUROC and AUPRC. Statistical significance was assessed using DeLong tests for AUROC comparisons and bootstrap confidence intervals. SHAP values were used for interpretability.

Key Results

1. Structured Digital Features Significantly Outperform Clinical Text

All structured digital models achieved AUROC > 0.90 (range 0.901–0.949), while all language models clustered between 0.886 and 0.892.

• Best Digital Model: BiLSTM at 48h (AUROC 0.949).
• Best Language Model: ClinicalBERT on discharge summaries (AUROC 0.892).
• Performance Gap: The best digital model outperformed the best language model by ΔAUROC = +0.057 (DeLong p < 0.001).
• Early Warning Capability: Even the weakest digital model (Transformer at 6h, AUROC 0.901) significantly exceeded the best language model (ΔAUROC = +0.009, p < 0.001), despite having access to only 3–5 laboratory measurements at that time point.

2. The "Information Ceiling" of Clinical Text

Text ablation experiments revealed a clear performance ceiling for language models.

• Adding discharge summaries to radiology reports (T2) yielded negligible improvement over radiology alone (T1).
• Discharge summaries alone (T3) performed slightly better (0.892) but did not surpass the ~0.892 saturation point regardless of text enrichment.
• This indicates that clinical text provides near-zero independent predictive value beyond what structured digital features capture for this specific condition.

3. Multimodal Fusion Yields Marginal Gains

Late-fusion multimodal models showed only marginal improvements over pure digital models.

• The maximum gain was ΔAUROC = +0.005 (p = 0.001) for XGBoost + full text.
• The prediction probabilities of digital and language models were highly correlated (Pearson r = 0.689), suggesting that text and numerical data describe the same underlying physiological state with high fidelity, rendering text informationally redundant in this context.

4. Interpretability and Clinical Alignment

SHAP analysis identified liver disease history (mean |SHAP| = 1.27) as the dominant predictor, followed by peak total bilirubin (0.27) and maximum PT (0.25). These findings align with AASLD diagnostic criteria and the pathophysiology of Acute-on-Chronic Liver Failure (ACLF), confirming the model learned genuine pathophysiological signals rather than statistical artifacts.

Significance and Claims

The paper claims to provide the first systematic comparison of structured digital features and clinical text for liver failure prediction on a unified cohort with identical labels and test sets, thereby eliminating confounding factors present in prior literature.

The study concludes that for conditions whose diagnosis is inherently anchored to objective numerical thresholds (such as liver failure defined by INR and bilirubin):

1. Structured digital features are superior: They substantially and significantly outperform clinical text across all metrics and time windows.
2. Text is redundant: Clinical text provides no independent predictive value beyond what digital features already capture, exhibiting a hard information ceiling (~AUROC 0.892) that cannot be overcome by text enrichment.
3. Design Implications: For numerically diagnosed conditions, clinical AI system design should prioritize structured digital models over language models. Language models represent a redundant technical choice for this specific class of problems, as they introduce noise from documentation style variation without adding orthogonal information.

The Clinical Transformer is highlighted as achieving state-of-the-art performance, capable of genuine early warning as early as 6 hours post-admission, outperforming all language models even at this earliest time point.