Psychometric evaluation of The Index of Myalgic Encephalomyelitis Symptoms (TIMES). Part II: Criterion-related and discriminant validity, test-retest reliability and minimal detectable difference.

This study validates the Index of Myalgic Encephalomyelitis Symptoms (TIMES) as a valid, reliable, and sensitive assessment tool for measuring symptom severity in ME/CFS patients, demonstrating strong criterion-related and discriminant validity, excellent test-retest reliability, and a defined minimal detectable difference.

The Gist

Imagine you are trying to describe a very complicated, invisible illness to a doctor. You have a headache, your brain feels foggy, your muscles ache, and you feel exhausted. But because there is no "blood test" or "X-ray" for Myalgic Encephalomyelitis (ME/CFS), doctors have to rely on you describing your symptoms.

The problem is that until now, the "rulers" we used to measure these symptoms were either too long (like a 100-question survey that takes forever to fill out) or hadn't been properly checked to see if they were accurate.

This paper is about testing a new, better ruler called TIMES (The Index of ME Symptoms). Think of TIMES as a high-tech, multi-tool Swiss Army knife designed specifically for ME/CFS.

Here is a simple breakdown of what the researchers did and what they found, using some everyday analogies:

1. The Goal: Building a Better Ruler

The researchers wanted to know: Does this new Swiss Army knife actually measure the illness correctly? Is it sturdy? And if I use it twice, do I get the same result?

They created TIMES with help from people who actually have the disease (co-production), ensuring it covers the real things patients care about, not just what doctors think is important. It breaks the illness down into 9 different "zones" (like Fatigue, Brain Fog, Pain, Sleep, etc.), rather than just giving one big score.

2. The Test: Does it Match Reality? (Validity)

To see if the ruler was accurate, they compared TIMES against another tool called the "ME Severity Scale" (a simple 1-to-5 rating of how sick someone feels).

• The Analogy: Imagine you have a new digital thermometer. To check if it works, you compare it to a trusted, old-school mercury thermometer. If the new one says "100°F" and the old one says "100°F," they match.
• The Result: The new TIMES thermometer matched the old severity scale very well. When people said they were "very sick," the TIMES score was high. When they were "less sick," the score was lower.
  • One small hiccup: The parts of the ruler measuring "stomach issues" and "immune system" didn't match as perfectly as the others. It's like if the thermometer was great at measuring fever but a little fuzzy when measuring body temperature in the morning. The researchers noted this but said it's still useful.

3. The Stress Test: Is it Sturdy? (Reliability)

They asked a group of people to fill out the survey, wait two weeks, and fill it out again. They wanted to see if the results stayed the same if the person's condition hadn't changed much.

• The Analogy: Imagine you weigh yourself on a scale. If you step on it, step off, and step back on 10 minutes later, the scale should show the same weight (give or take a tiny bit). If the scale jumps from 150lbs to 200lbs randomly, it's broken.
• The Result: The TIMES scale was very sturdy. Most of the "zones" gave consistent results. Two zones (Cranial Nerves and Immune System) were "good" but not "excellent," likely because those symptoms can be a bit more jumpy or hard to pin down.

4. The Sensitivity: Can it Spot Small Changes? (Minimal Detectable Difference)

This is a fancy way of asking: If a patient gets slightly better or slightly worse, will the tool actually notice, or will it just say "same as before"?

• The Analogy: Imagine a ruler that only has marks for every 10 inches. If you grow from 5'0" to 5'1", that ruler won't show the change. You need a ruler with marks for every inch to see the growth.
• The Result: TIMES is a very precise ruler. It can detect small changes in how a person feels. This is crucial for doctors to know if a treatment is working or if a patient is getting worse, even if the change isn't huge.

5. The "Fluctuation" Problem

The researchers admitted something important: ME/CFS is a rollercoaster. A person might feel terrible one day and "okay" the next, even if their overall condition hasn't changed.

• The Analogy: Imagine trying to measure the water level in a bucket that is being splashed by a wave. Sometimes the water is high, sometimes low. If you take a snapshot, it might look different than the next snapshot, even if the bucket hasn't changed size.
• The Takeaway: Because the illness fluctuates so much, the scores might wiggle a bit. The researchers suggest that in the future, we might need to ask patients about their symptoms over a shorter time (like one week instead of two) to get a clearer picture.

The Bottom Line

The TIMES tool is a valid, reliable, and sensitive way to measure the symptoms of ME/CFS.

• Why it matters: It's free to use, it's shorter than the old 100-question surveys, and it gives doctors a detailed "map" of a patient's specific symptoms (e.g., "Your pain is high, but your sleep is okay").
• The Verdict: It's a solid, well-tested tool that helps turn the "invisible" suffering of ME/CFS into clear, measurable data that can help improve care and research.

Technical Summary

1. Problem Statement

Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS) is a complex, disabling condition with no known biomarkers, necessitating diagnosis based on symptom-based criteria. Currently, there is a scarcity of validated psychometric tools to assess the specific symptom burden of ME/CFS.

• Limitations of Existing Tools: The primary existing tool, the DePaul Symptom Questionnaire (DSQ), is criticized for its excessive length (99 items), lack of reported correlation coefficients in validity studies, absence of Minimal Detectable Difference (MDD) data, and lack of construct validity analysis using Item Response Theory (IRT).
• Clinical Need: There is an urgent need for a concise, psychometrically robust, and clinically feasible assessment tool that can characterize symptom profiles for clinical practice, service development, and research, rather than solely for diagnostic criteria operationalization.

2. Methodology

The study employed a "classic theory" approach to evaluate the psychometric properties of the TIMES (The Index of ME Symptoms), a tool co-produced with patients and clinicians.

• Study Design & Participants:

  • Sample: 1,055 participants with a formal diagnosis of ME/CFS in the UK were recruited via the ME Association and support groups.
  • Cohorts: Three cohorts were utilized:
    1. TIMES 1 (n=701): Initial iteration testing.
    2. TIMES 2 (n=354): Revised version testing.
    3. Retest Cohort (n=100): A subset of participants who reported their condition was "the same" over a two-week period, used for reliability testing.
  • Demographics: Predominantly middle-aged women with moderate-to-severe disease duration (mean ~15 years).

• Instrument:

  • The TIMES assesses 58 symptoms across nine domains (Fatigue, Cognition, Pain, Sleep, Motor-sensory, Cardio-respiratory, Gastro-intestinal, Cranial nerves, Immune system) using a 4-point Likert scale (0–3).
  • It generates a Total Score (TIMES-Total) and domain-specific sub-scores.

• Data Transformation & Analysis:

  • Rasch Transformation: Raw ordinal scores were converted to 0–100 interval-level data using logit-based transformation. This allows for the use of parametric statistics and increases measurement precision.
  • Validity:
    • Criterion-related (Convergent): Pearson correlations with the ME/CFS Severity Scale (a 5-level ordinal scale).
    • Discriminant: Analysis of Variance (ANOVA) to test score differences across the five severity levels.
  • Reliability:
    • Test-Retest: Intra-class Correlation Coefficients (ICC) using a two-way random effects model.
  • Sensitivity: Minimal Detectable Difference (MDD) calculated from the Standard Error of Measurement (SEM) of the ICC.

3. Key Contributions

• Development of TIMES: Introduction of a comprehensive, co-produced symptom inventory specifically designed for clinical assessment and monitoring in ME/CFS.
• Methodological Rigor: Application of Rasch analysis to transform ordinal data into interval data, a novel approach in ME/CFS symptom measurement that enhances statistical power and allows integration with continuous biomedical data.
• Comprehensive Psychometric Evaluation: The study provides the first full suite of psychometric data for an ME/CFS symptom scale, including criterion validity, discriminant validity, test-retest reliability, and MDD, adhering to COSMIN guidelines.
• Accessibility: The tool is freely available for clinical and research use via the ME Association website and app.

4. Key Results

• Criterion-Related Validity:
  • Significant moderate correlations ($r = 0.400 - 0.528, p < 0.001$) were found between TIMES scales and ME/CFS severity.
  • The Fatigue scale showed the strongest correlation ($r=0.528$).
  • Gastro-intestinal ($r=0.315$) and Immune system ($r=0.302$) scales showed lower but still significant correlations.
• Discriminant Validity:
  • Significant differences in scores were observed between all five levels of ME/CFS severity for most scales ($p < 0.001$).
  • Some comparisons between Level 4 (Severe) and Level 5 (Very Severe) were non-significant, attributed to small sample sizes (underpowered) in these groups.
• Test-Retest Reliability:
  • Excellent: Most scales demonstrated ICC > 0.70 (e.g., Total Score ICC = 0.795, Neurology = 0.773).
  • Good: Cranial nerves (ICC = 0.681) and Immune system (ICC = 0.669) scales showed good reliability.
• Minimal Detectable Difference (MDD):
  • MDD values ranged from 3.95% to 17.45% on the 0–100 scale.
  • The Total TIMES score had the lowest MDD (3.95%), indicating high sensitivity to change.

5. Significance and Implications

• Clinical Utility: The TIMES provides clinicians with a detailed, validated profile of symptom burden, enabling better characterization of individual patient needs beyond simple diagnosis.
• Research Advancement: The use of Rasch-transformed interval data allows researchers to perform parametric analyses and correlate symptom data with physiological or biomedical markers, potentially advancing the understanding of ME/CFS pathophysiology.
• Monitoring Change: The calculation of MDD allows clinicians and researchers to distinguish between true clinical improvement/deterioration and measurement noise, which is crucial for evaluating treatments in a fluctuating condition.
• Comparison to DSQ: While the DSQ remains a diagnostic tool, TIMES offers superior psychometric transparency (reported coefficients, MDD) and a more feasible format for routine clinical monitoring.

Limitations Noted: The study acknowledges potential selection bias toward English-speaking, socially active patients and the lack of confirmed medical records for diagnoses. Additionally, the fluctuating nature of ME/CFS symptoms may impact the stability of test-retest reliability over a two-week period.

Conclusion: The TIMES scales are validated, reliable, and sensitive instruments for assessing ME/CFS symptoms, filling a critical gap in the psychometric toolkit for this condition.