Predicting Trajectories of Long COVID in Adult Women: The Critical Role of Causal Disentanglement

1. Problem Statement

The paper addresses the critical challenge of predicting the severity of Post-Acute Sequelae of SARS-CoV-2 (PASC), commonly known as Long COVID, specifically in adult women. The core difficulty lies in the "signal-to-noise" problem:

• Diagnostic Overlap: Hallmark PASC symptoms (e.g., fatigue, insomnia, joint pain, cognitive fog) significantly overlap with symptoms caused by natural biological transitions (specifically menopause) and common comorbidities (e.g., diabetes, cardiovascular issues).
• Confounding Bias: Standard predictive models often conflate these baseline physiological changes ("noise") with active viral pathology ("signal"), leading to diagnostic ambiguity, over-diagnosis, and biased treatment pathways.
• Data Complexity: While wearable technology provides rich longitudinal data (cardiac, sleep, activity), the high dimensionality and inherent noise require advanced computational methods to distinguish true causal disease markers from baseline physiological fluctuations.

2. Methodology

The authors propose a Causal-Disentangled Regressor built upon a Large Language Model (LLM) backbone (specifically Qwen-2.5-0.5B) to isolate invariant disease features from confounding variables.

Data Sources

• Cohort: 1,155 adult women (median age 61) from the NIH RECOVER dataset.
• Multi-Modal Input:
  1. Static Profiles: Demographics and clinical comorbidities (cardiovascular, mental health, menopause status).
  2. Longitudinal Wearable Data: Four weeks of mean-aggregated physiological signals, including Heart Rate Variability (HRV), respiratory metrics, sleep architecture (latency, REM), and physical activity intensity.
  3. Target: Future PASC severity scores (3-month prediction).

Architectural Framework

The model employs a specialized architecture designed to bifactorize the latent representation ($h$) of the input narrative:

1. Attention-Based Feature Disentanglement:

   • A Disentanglement Attention Layer uses a Softmax-MLP mechanism to generate scalar gates ($\alpha_o$ for causal, $\alpha_c$ for confounding).
   • The latent state is projected into two distinct vectors:
     • Causal Feature ($x_o$): Captures invariant symptoms directly linked to PASC pathology (e.g., "malaise," "brain fog").
     • Confounder Feature ($x_c$): Isolates noisy variables (e.g., "menopause," "diabetes") and linguistic artifacts.

2. Environment Mixing for Robustness:

   • To prevent reliance on spurious correlations, the model generates synthetic counterfactual representations ($x_{mix}$) during training.
   • It combines the causal features of patient $i$ with the confounding features of a randomly sampled patient $j$ ($x_{mix} = \text{Combine}(x_{o,i}, x_{c,j})$).
   • This forces the regression head to predict PASC scores solely based on the causal signal, regardless of the confounding environment.

3. InfoNCE-based Contrastive Mix Loss:

   • A specialized loss function ($L_{mix}$) based on Information Noise Contrastive Estimation (InfoNCE) is used.
   • It aligns the mixed representation with the original clinical graph to ensure semantic consistency, mathematically enforcing the independence of disentangled features and ensuring causal information remains the dominant signal.

4. Multi-Head Regression:

   • The model utilizes three prediction heads: a Causal Head ($f_o$), a Confounder Head ($f_c$), and a Joint Head ($f_{co}$).
   • All heads are supervised against the ground-truth PASC score, training the network to identify which input components are truly predictive versus those characteristic of the baseline state.

3. Key Results

The study utilized a multi-seed validation strategy (5 random seeds) with an 80/20 train/test split.

• Performance Comparison (vs. XGBoost Baseline):

  • XGBoost: Achieved superior regression metrics with low variance (RMSE: $4.76 \pm 0.16$, MAE: $3.51 \pm 0.08$). It is a reliable predictor for raw score estimation but lacks causal interpretability.
  • Causal Inference Network: Achieved competitive classification accuracy ($0.867 \pm 0.044$) and superior precision ($0.836 \pm 0.116$). While it showed higher variance in regression metrics (RMSE: $5.35 \pm 1.17$), this is a deliberate trade-off resulting from the objective function penalizing environmental confounders.

• Saliency Analysis (Causal Attribution):

  • Active Pathology: Tokens representing direct indicators like "breathlessness" and "malaise" achieved maximum saliency (1.00), confirming the model's focus on pulmonary and systemic dysfunction.
  • Baseline Noise Suppression: Confounding factors such as "menopause" and "diabetes" were successfully suppressed with saliency scores below 0.27.
  • Linguistic Noise: Administrative tokens (e.g., "that," "to") were ignored (saliency < 0.17).

4. Key Contributions

1. Causal Disentanglement in Clinical LLMs: The paper introduces a novel architecture that explicitly partitions latent space to separate causal disease markers from environmental/biological confounders, addressing a major limitation in current PASC research.
2. Integration of Wearable and LLM Data: It demonstrates a framework for fusing structured clinical histories with unstructured/semi-structured longitudinal wearable data within a semantic narrative format.
3. "Clinically Honest" Prediction: The model prioritizes precision and interpretability over raw regression stability. By filtering out baseline noise (like menopause), it reduces the risk of over-diagnosis and misattribution of symptoms.
4. Environment Mixing Strategy: The use of counterfactual data augmentation (mixing causal features of one patient with confounders of another) effectively trains the model to be invariant to specific patient backgrounds.

5. Significance

This research offers a robust framework for sex-specific and endocrine-aware diagnostic tools. By successfully distinguishing between viral pathology and hormonal transitions (menopause), the proposed method provides a more accurate assessment of Long COVID trajectories in women, a demographic with the highest risk profiles. The approach transforms standard LLMs into clinically aware diagnostic instruments that can guide personalized treatment pathways by focusing on true causal drivers of disease severity rather than statistical correlations with baseline health conditions.