Compact longitudinal representations derived from mixed-format lifestyle questionnaires outperform static text-derived features for ALS-versus-control classification

This study demonstrates that in small clinical cohorts, compact longitudinal representations of lifestyle questionnaire data, particularly those capturing temporal changes, significantly outperform static text-derived features for distinguishing ALS patients from controls.

The Gist

Imagine you are trying to figure out who is sick and who is healthy by looking at a patient's life story. You have two types of information: a checklist (structured data like age, BMI, and yes/no questions) and a diary (free-text answers where patients write about their habits, feelings, and history in their own words).

For a long time, researchers thought: "If we just get a super-smart AI to read all those diary entries and turn them into more checklist items, we'll get a much better prediction!"

This paper is like a reality check that says: "Not so fast. It's not about adding more items to the list; it's about how you tell the story of change over time."

Here is the breakdown of what they did and what they found, using some simple analogies.

The Setup: The "Life Questionnaire"

The researchers had 103 people (some with ALS, a serious nerve disease, and some healthy controls). They gave them a questionnaire with two parts:

1. The Checklist: Standard questions with fixed answers.
2. The Diary: Open questions like "Tell us about your sports habits" or "Describe your diet."

They collected this data at two different times in the patients' lives (Time 1 and Time 2).

The Experiment: Three Different "Lunch Boxes"

The team tried to build a computer model to predict who had ALS using three different "lunch boxes" of data:

• Box 1 (The Baseline): Just the checklist. (The "Boring" option).
• Box 2 (The Static Diary): The checklist + the diary entries from Time 1, turned into checklist items by an AI. (The "More Info" option).
• Box 3 (The Story of Change): The checklist + the diary from Time 1 + a summary of how things changed between Time 1 and Time 2. (The "Trajectory" option).

The Big Mistake: The "Leaky Bucket"

Before they got the real results, they realized their previous experiments had a "leak." Imagine trying to measure how much water a bucket holds, but you accidentally left the lid off, and some water from the future (the test data) was dripping into the past (the training data).

When they fixed this "leak" (by making sure the computer never peeked at the test answers while learning), the scores dropped. This was actually a good thing! It meant their previous scores were fake and too optimistic. Now, they were seeing the real performance.

The Results: What Actually Worked?

1. The "More Info" Strategy Failed (Box 2)
They thought that just adding the AI-read diary entries from the first time point would help.

• The Analogy: It's like trying to guess if a car is going to break down by adding 500 extra details about the color of the seats and the brand of the radio.
• The Result: It didn't help much. The computer got confused by all the extra static details, and the prediction didn't get better. The diary entries from the first time point were mostly redundant; the checklist already told the computer most of what it needed to know.

2. The "Story of Change" Strategy Won (Box 3)
The magic happened when they stopped just listing facts and started describing movement. Instead of saying "Patient A weighs 70kg," they said "Patient A gained 5kg and stopped running between Time 1 and Time 2."

• The Analogy: Imagine trying to predict the weather.
  • Static approach: "It is 70°F right now." (Not very helpful for a storm).
  • Change approach: "The temperature dropped 20 degrees in one hour, and the wind just picked up." (This tells you a storm is coming!).
• The Result: This "Change Box" (Pool 3) was the clear winner. The computer model (specifically a Random Forest) got significantly better at spotting the ALS patients.

The "Ablation" Test: Taking the Engine Out

To be sure, they did a "surgery" on their best model. They took out the "Diary" part and the "Change" part separately to see which one was the engine.

• Taking out the Diary: The car still ran fine. (The static text wasn't the secret sauce).
• Taking out the Change: The car stopped dead. (The "Change" data was the only thing that made the model work).

The Big Takeaway

The main lesson of this paper is a shift in how we use AI in medicine:

Don't just use AI to turn more words into more numbers. In small groups of patients, adding more static facts often just creates noise.

Instead, use AI to summarize the journey. The real power of reading a patient's diary isn't to find new facts, but to compress their life story into a clear picture of how they are changing.

In simple terms:
If you want to know if a car is broken, don't just list every screw and bolt (static data). Look at the odometer and ask: "How fast is the speedometer dropping?" (Longitudinal change). That's where the real answer lies.

Technical Summary

1. Problem Statement

The study addresses a critical gap in machine learning (ML) applications for Amyotrophic Lateral Sclerosis (ALS) classification. While ML models in ALS typically rely on structured biomedical signals or neuroimaging, patient-reported lifestyle data is often collected via mixed-format questionnaires containing both structured variables and free-text narratives.

• The Challenge: It remains unclear whether extracting variables from free-text responses adds predictive value beyond existing structured data, particularly in small clinical cohorts where adding features can lead to overfitting.
• The Hypothesis: The authors hypothesized that the predictive value of language-based processing depends on how the information is represented. Specifically, they tested whether compact longitudinal summaries of change (trajectories) outperform static text-derived features (enrichment of a single time point).
• Methodological Concern: Previous studies often suffer from data leakage (e.g., preprocessing steps applied before train-test splitting), leading to overly optimistic performance estimates.

2. Methodology

Data and Cohort

• Source: Retrospective data from Hospital Clínico San Carlos, Madrid.
• Cohort: 103 participants (60 controls, 43 ALS cases; ALS-FTD cases were relabeled as ALS).
• Data Structure: Mixed-format lifestyle questionnaires collected at two time points:
  • T1: Earlier life (defined by age: <35 for those <65; <65 for those ≥65).
  • T2: Later/current life (post-35 or post-retirement).
  • Content: Structured variables (demographics, BMI, habits) and free-text fields (work, sport, diet, social life).

Feature Engineering & Pipeline

The study developed a leakage-free machine learning pipeline with three distinct feature configurations (Pools):

1. Pool 1 (Baseline): Structured baseline variables only.
2. Pool 2 (Static Enrichment): Pool 1 + compact summaries derived from T1 free-text responses.
3. Pool 3 (Longitudinal Enrichment): Pool 2 + compact descriptors of change between T1 and T2.

Key Technical Steps:

• LLM-Based Extraction: Used GPT-4o mini via a schema-guided prompt to extract structured JSON updates from free-text. The model acted as a "conservative data assistant," filling missing structured values only when explicit text evidence existed.
• Compact Representation:
  • Text: Individual habits were aggregated into 5 semantic groups (e.g., Endurance, Gym, Mind-Body) and summary counts (e.g., TXT_HABIT_COUNT).
  • Longitudinal: Instead of adding raw T2 variables, the pipeline computed delta variables (changes) and domain-level change counts between T1 and T2. This reframed the data from "state" to "trajectory."
• Preprocessing: Strictly embedded within the training folds to prevent leakage. Steps included missingness filtering, imputation (median/most frequent), standardization, and dimensionality reduction (removing high correlation >0.90).

Models and Evaluation

• Classifiers: Logistic Regression (Elastic Net), Linear Support Vector Classification (SVC), and Random Forest (RF).
• Validation Strategy:
  • Repeated Stratified Holdout: 10 seeds, 75/25 split.
  • Repeated Stratified 5-Fold Cross-Validation: 10 seeds.
  • Hyperparameter Tuning: Inner 3-fold CV within each training split.
• Metrics: Accuracy, Weighted F1, ALS Recall, Balanced Accuracy, and Matthews Correlation Coefficient (MCC).
• Ablation Study: Performed on the best models to isolate the contribution of the "compact text block" vs. the "compact temporal block."

3. Key Results

Impact of Leakage Correction

Initial analyses showed optimistic results. Once the pipeline was corrected for data leakage (moving all preprocessing inside the training loop), performance estimates dropped significantly, indicating previous results were inflated. The corrected metrics provided a more realistic baseline.

Performance Comparison (Holdout & CV)

• Pool 1 (Structured only): Baseline performance (RF: Acc ~0.58, F1w ~0.58).
• Pool 2 (Static Text): Adding T1 text summaries did not provide a robust improvement over Pool 1. In some cases, performance slightly decreased or remained stagnant.
• Pool 3 (Longitudinal Change): Achieved the best performance.
  • Random Forest (Pool 3): Accuracy 0.673, Weighted F1 0.666, MCC 0.323 (Holdout).
  • Cross-validated F1: 0.654; MCC: 0.312.
  • This represented a clear separation from Pool 1 and Pool 2.

Ablation Analysis (Crucial Finding)

• Removing Text Block (in Pool 3): Had little to no negative effect on performance (F1w actually increased slightly from 0.666 to 0.683 in holdout).
• Removing Temporal Block (in Pool 3): Caused a drastic performance drop.
  • F1w fell from 0.666 to 0.542.
  • MCC fell from 0.323 to 0.066.
  • Confusion matrices showed a massive increase in False Negatives.

Feature Dimensionality

Despite adding T2 information, the "compact longitudinal" approach in Pool 3 actually reduced the effective number of retained features (from ~213 to ~148) compared to a naive expansion, proving that the gain came from information density and trajectory representation, not feature volume.

4. Key Contributions

1. Methodological Rigor: Demonstrated the critical importance of leakage-free pipelines in small clinical datasets, showing that uncorrected preprocessing leads to misleadingly high performance.
2. Representation over Extraction: Challenged the assumption that NLP/LLMs are primarily useful for expanding static feature spaces. The study proves that in small cohorts, the value of LLMs lies in abstraction and summarization of longitudinal trajectories.
3. Longitudinal Signal: Identified that the predictive signal for distinguishing ALS from controls in lifestyle data resides in how patients change over time (T1 to T2), not in static snapshots of their habits.
4. Ablation Evidence: Provided empirical evidence that static text-derived variables from a single time point are often redundant with structured data, whereas compact descriptors of change are the primary drivers of classification performance.

5. Significance and Implications

• For Clinical ML: The study suggests that in small datasets, "more features" (via NLP) is not the solution; "better representation" (via longitudinal encoding) is. Future work should prioritize trajectory-aware representations over static feature enrichment.
• For NLP/LLM Application: LLMs should be utilized not just to extract entities, but to synthesize complex, time-dependent narratives into compact, model-compatible descriptors of change.
• For ALS Research: Highlights that lifestyle trajectories (e.g., changes in physical activity, diet, or socialization over decades) may contain distinct phenotypic signals for ALS that static questionnaires miss.
• Limitations: The study is limited by a small sample size (N=103), lack of external validation, and binary classification (ALS vs. Control) rather than subtyping. However, the methodological findings regarding representation design remain robust.

Conclusion: The paper concludes that the main predictive gain in this setting did not arise from static text-derived variables, but from representing questionnaire information as compact longitudinal change descriptors. This shifts the paradigm for NLP in clinical ML from "feature expansion" to "trajectory synthesis."