Eliciting associations between clinical variables from LLMs via comparison questions across populations

Technical Summary: Eliciting Associations between Clinical Variables from LLMs via Comparison Questions

Problem Statement
Large Language Models (LLMs) are trained on vast biomedical literature encompassing diverse patient populations, potentially encoding complex correlation and causal structures between clinical variables. However, directly querying LLMs for specific statistical relationships (e.g., "What is the correlation between X and Y in population Z?") is fraught with pitfalls. Direct elicitation often yields answers based on specific recall rather than aggregated knowledge, suffers from sycophancy (adjusting to perceived user beliefs), is sensitive to prompt ordering, and may produce hallucinated or post-hoc rationalized outputs. Furthermore, LLMs cannot be treated as self-reflecting agents; they must be treated as experimental subjects whose internal representations are inferred through behavior. The challenge is to recover meaningful association structures and potential causal links from LLMs without access to their internal activations or weights, while accounting for population-specific variations.

Methodology
The authors propose a framework that treats the LLM as an experimental subject, using structured comparison questions to infer implicit correlations and applying Invariant Causal Prediction (ICP) to identify robust causal candidates.

1. Triplet Comparison Questions:
   Instead of asking for a numeric correlation, the method queries the LLM with triplet questions involving three artificial patients (1, 2, and 3).

   • Setup: For a pair of variables $X_j$ and $X_k$, the values of $X_j$ are specified for all three patients. The value of $X_k$ is specified only for Patient 3.
   • Task: The LLM is asked whether Patient 3 is more similar to Patient 1 or Patient 2, taking the value of $X_k$ into account.
   • Mechanism: By systematically varying the values of $X_k$ for Patient 3 while holding the reference patients constant, the method observes how the "balance" of the similarity decision shifts. If $X_j$ and $X_k$ are strongly correlated in the LLM's representation, the value of $X_k$ will significantly influence the decision.

2. Surrogate Decision Model:
   The binary responses (Patient 1 vs. Patient 2) are modeled using a statistical surrogate. The authors assume a "rational" LLM representation where the model performs an inverse-variance estimation to predict the true state of a patient based on noisy measurements and indirect cues.

   • The probability of choosing Patient 2 is modeled as a logistic function of the difference between the estimated state of Patient 3 and a reference midpoint.
   • The fitted logistic coefficients ($\beta$) are mathematically related to the underlying population correlation ($\rho$).
   • Symmetric Estimator: To eliminate nuisance terms related to variable scaling and relative cue weighting, the method flips the roles of $X_j$ and $X_k$ in a second set of queries. The correlation is recovered from the product of the slope ratios of the two directional queries, yielding a symmetric estimate $\hat{\rho}$.

3. Prompted Environment Shifts and ICP:
   To move from correlation to causal inference, the method employs Invariant Causal Prediction (ICP).

   • Synthetic Environments: The authors create different "environments" by varying the prompt text to specify different subpopulations (e.g., different mean values for specific variables like smoking history or air pollution exposure).
   • ICP Application: Correlation matrices are estimated for each prompted environment. ICP tests subsets of candidate predictors to see if their regression slopes on a target variable remain invariant across these environments. Spurious correlations typically break under distribution shifts, while true causal links remain stable.
   • Conservative Filtering: A variable is retained as a candidate parent only if it cannot be excluded from any invariant set, ensuring a conservative set of causal candidates.

Key Contributions

• Triplet-based Extraction: A novel framework for recovering pairwise correlations from LLM similarity judgments without requiring access to model internals (activations or weights).
• Prompted Environments for Causal Assessment: A method to estimate environment-specific correlations by manipulating prompt descriptions of subpopulations, enabling the application of ICP to LLM-derived data.
• Clinical Demonstration: Validation in two distinct clinical domains: Chronic Obstructive Pulmonary Disease (COPD) and Multiple Sclerosis (MS).

Results
The method was evaluated using OpenAI's gpt-oss-120b and gpt-oss-20b models on COPD (lung function markers) and MS (biomarkers and clinical scales) datasets.

• Model Fit and Stability: The surrogate decision model fit the binary triplet responses well. The decision surfaces were smooth and monotone, with decision boundaries consistent with expected correlation signs (e.g., negative slopes for positively correlated pairs).
• Clinical Plausibility:
  • COPD: The extracted correlations showed strong, coherent structure. Spirometry variables (FEV1, FEV1/FVC) formed a tight block, and known physiological relationships (e.g., negative association between FEV1/FVC and air trapping metrics) were recovered.
  • MS: Correlations were generally weaker than in COPD, likely due to sparser or more heterogeneous literature in the training data. However, established pairs (e.g., T25FW and T2-LV) were recovered.
• Comparison with Direct Queries: Triplet-based estimates were significantly less noisy and more stable than direct numeric queries. Direct queries produced dispersed distributions (standard deviations 0.1–0.3) and were less sensitive to environment shifts, whereas triplet estimates varied appropriately across environments, a prerequisite for ICP.
• Causal Inference (ICP):
  • COPD: ICP identified a small set of conservative candidate parents: pack-years $\to$ DLCO, PM2.5 $\to$ TLC, and BMI $\to$ R5-R20. These align with established clinical knowledge (e.g., smoking reducing diffusion capacity).
  • MS: ICP identified T1-black-hole-volume as a parent of EDSS and sGFAP as a parent of 9HPT, consistent with meta-analytic evidence on lesion burden and disability.
• Robustness: Estimates were stable across different reference value choices and showed consistent relative structure across model sizes (120b vs. 20b), though the smaller model yielded slightly lower absolute correlation magnitudes.

Significance and Claims
The paper claims that indirect elicitation via triplet comparisons offers a "cautious route" from implicit correlations in LLMs to causal statements that are congruent with the model's answering patterns. The significance lies in:

1. Overcoming Direct Query Limitations: The method avoids the hallucination and sycophancy associated with direct numeric queries by relying on relative similarity judgments.
2. Behavioral Probing: It treats LLMs as experimental subjects, inferring their internal representation of clinical relationships through controlled behavioral experiments rather than assuming they can self-report truth.
3. Causal Discovery without Data: It demonstrates a pathway to perform invariant causal prediction using only prompt-engineered "synthetic" environments, bypassing the need for multiple individual-level clinical datasets which are often difficult to obtain.

The authors remain modest, noting that the method recovers model-implied associations rather than empirical clinical truths. The translation from fitted slopes to correlations relies on specific relational assumptions, and the study is limited to two domains and a specific set of models. Nevertheless, the framework provides a viable exploratory tool for probing the association structure represented within LLMs.