Reproducibility and incremental predictive value of pattern and constitution diagnosis in chronic musculoskeletal pain: a scoping review

Technical Summary: Reproducibility and Incremental Predictive Value of Pattern and Constitution Diagnosis in Chronic Musculoskeletal Pain

Problem Statement
The clinical logic of East Asian medicine (including Traditional Korean Medicine, Traditional Chinese Medicine, and Kampo) relies on stratifying patients via "pattern identification" (syndrome differentiation) and "constitution diagnosis" to individualize treatment. For this logic to be scientifically valid, two premises must hold: (1) the diagnosis must be reproducible across different raters, and (2) the resulting classification must predict treatment response or prognosis better than established methods.

Despite the inclusion of traditional medicine conditions in ICD-11, the evidence chain connecting reproducible diagnosis to outcome-anchored prediction remains unmapped. Previous literature suggests inter-rater reproducibility is often low-to-moderate, and direct evidence that pattern-based individualization outperforms fixed treatment is inconsistent. Furthermore, while risk-stratification tools exist in mainstream pain medicine (e.g., STarT Back, Keele STarT MSK), it is unclear whether pattern/constitution diagnoses provide incremental predictive value beyond these established clinical and psychosocial predictors, particularly for chronic low back and neck pain.

Methodology
This study is a scoping review conducted following the Arksey and O'Malley framework, Levac et al.'s enhancements, and updated JBI guidance, reported per PRISMA-ScR. The review was registered on the Open Science Framework.

• Search Strategy: Two distinct search strings were developed and iteratively refined:
  • Track A (Reproducibility): Deliberately disease-agnostic, searching for inter-rater reliability studies of pattern/constitution diagnosis.
  • Track B (Incremental Prediction): Focused on adults with chronic musculoskeletal pain (low back, neck, shoulder, knee osteoarthritis), searching for studies evaluating whether pattern/constitution adds predictive value beyond established stratification tools.
• Databases: PubMed/MEDLINE, Cochrane CENTRAL, KMBASE, and OASIS.
• Eligibility: Included primary studies, protocols, and methodological studies in English and Korean (with limited screening of Chinese/Japanese sources). Excluded pure efficacy studies without diagnostic components, animal studies, and reviews (used only for background).
• Data Charting: Items extracted included diagnostic procedures, reproducibility statistics (Cohen's kappa, Gwet's AC, ICC), comparator tools, incremental-value metrics (discrimination, reclassification, calibration, decision-curve analysis), external validation, and pre-specified Minimal Clinically Important Difference (MCID).
• Synthesis: Effect estimates were not pooled. Instead, the literature was mapped to identify ranges, types, and gaps across the two tracks.

Key Results
From an initial 211 records, 76 studies were included after screening and de-duplication: 42 for Track A (reproducibility) and 34 for Track B (prediction).

• Track A: Reproducibility

  • Reproducibility of pattern and constitution diagnosis is generally low-to-moderate.
  • A previous systematic review cited in the data reported a mean Cohen's kappa of approximately 0.34 across seven studies.
  • Unstructured diagnosis showed poor reliability, whereas structured instruments improved agreement (e.g., kappa 0.56 in prediabetes).
  • Chance-corrected statistics (e.g., Gwet's AC2) yielded higher estimates (approx. 0.71) than uncorrected metrics.
  • Korean data on Sasang constitution and stroke four-examination diagnosis formed a significant cluster, showing modest expert agreement (kappa 0.248–0.501).

• Track B: Incremental Prediction

  • The evidence linking pattern/constitution to treatment response is heavily skewed toward Rheumatoid Arthritis (19/34 studies), mostly at the level of correlation or classification accuracy.
  • Coverage for primary conditions of interest was sparse: Low Back Pain (2 studies) and Neck Pain (1 study).
  • Critical Gap: No eligible peer-reviewed study identified in this search directly evaluated the incremental predictive value of pattern/constitution diagnosis beyond established pain-stratification tools (e.g., STarT Back, Oerebro/OMPSQ, treatment-based classification).
  • Existing studies often lacked external validation, calibration, decision-curve analysis, or pre-specified MCID.
  • Benchmark data indicates that even mainstream machine learning models for musculoskeletal pain prediction struggle with discrimination (AUC 0.49–0.65), suggesting the task is inherently difficult.

Key Contributions

1. Evidence Mapping: The review provides a structured map of the literature, separating reproducibility (Track A) from predictive validity (Track B), revealing that the "evidence chain" is broken.
2. Identification of the "Incremental-Prediction Gap": The review explicitly identifies that while correlations exist (mostly in Rheumatoid Arthritis), no study has demonstrated that pattern/constitution variables add predictive information over standard clinical and psychosocial predictors in chronic musculoskeletal pain.
3. Methodological Critique: The paper highlights recurring limitations in the field, including "circularity" (using expert consensus as a gold standard despite low reproducibility) and the absence of benchmarking against established tools.

Significance and Proposed Research Sequence
The paper concludes that pattern/constitution diagnosis currently shows limited reproducibility, and individualization has not been proven superior to standard treatment. Specifically for chronic low back and neck pain, the incremental predictive value of these diagnoses beyond established tools has not been evaluated.

The authors propose a specific research sequence to address these gaps:

1. Standardize Diagnosis: Prioritize the use of structured tools, training, and chance-corrected statistics to ensure reproducibility.
2. Anchor Validity: Evaluate validity against clinical outcomes (treatment response, adverse events) and pre-specified MCIDs, rather than expert consensus alone.
3. Evaluate Incremental Value: Rigorously test whether adding pattern/constitution variables to a baseline model (containing established tools like STarT Back and standard psychosocial predictors) improves discrimination, reclassification, and calibration.

This approach reframes the scientific evaluation of East Asian medicine from broad efficacy claims to testable questions regarding reproducibility, outcome-anchored prediction, and incremental clinical utility.