Electronic health record (EHR)-detectable statin intolerance phenotypes: Prevalence and validation in real-world general practice

This study utilized the ePBRN dataset to identify and validate six EHR-detectable phenotyping algorithms for statin intolerance, finding a 5.09% prevalence rate and concluding that while these tools offer valuable decision support, they should complement rather than replace clinical judgment and patient engagement in diagnosis.

The Gist

The Big Picture: The "Statin Intolerance" Mystery

Imagine statins as a highly effective "bodyguard" for your heart. They are pills that lower cholesterol and stop heart attacks. Most people can take them without any trouble. However, for some people (about 5% to 15%), these bodyguards cause problems. They might cause muscle aches, weakness, or other side effects. This is called Statin Intolerance.

The problem is that "intolerance" is a bit like a ghost. It's hard to catch. Some people feel pain that is real; others might feel pain because they are worried about the pill (a psychological effect called the nocebo effect). Because it's so hard to define, doctors and scientists have been arguing about how to count these cases.

The Mission: Building a "Digital Detective"

The researchers in this study wanted to solve a mystery: How many people in a real-world doctor's office actually have statin intolerance?

They didn't want to rely on doctors guessing or patients remembering their symptoms from years ago. Instead, they wanted to use Electronic Health Records (EHRs). Think of the EHR as a massive, digital filing cabinet containing every note, lab test, and prescription for thousands of patients.

Their goal was to build "Digital Detectives" (called algorithms) that could scan these digital files and automatically flag patients who might be intolerant to statins.

The Experiment: The "Six Detectives"

The researchers didn't just build one detective; they built six different ones, each based on rules from different parts of the world (Minnesota, Japan, Singapore, the USA, and the UK).

Think of these six detectives as having different rulebooks:

• Detective A (Minnesota): "If the patient stopped the pill and their muscle enzyme levels were high, they are intolerant."
• Detective B (Japan): "If the patient has muscle pain and a specific enzyme spike, they are intolerant."
• Detective C (Singapore): "If they have muscle pain or a high enzyme level, they are intolerant."

Each detective had slightly different criteria for what counts as "guilty" (intolerant) and what counts as "innocent" (tolerant).

The Test: The "Gold Standard" Trial

To see which detective was the best, the researchers created a "Gold Standard" group. They took a small sample of patients and had two human experts manually read through their entire medical history to decide, one by one, who was truly intolerant based on strict medical rules. This was the "Truth."

Then, they let the six Digital Detectives scan the same group and compared their results to the Human Truth.

The Results: The "Low Prevalence" Surprise

1. The Count:
When they looked at the whole group of patients (over 15,000 people), they found that 5.09% were statin intolerant.

• The Analogy: Imagine a stadium full of 100 people. Only about 5 of them are actually having trouble with the statin. This is on the lower end of what experts expected (which was usually 5–15%).
• Why so low? The researchers think it's because they used very strict rules to find these people. Also, some people who say they are intolerant might just be worried about the pill, not actually having a physical reaction.

2. The Detective Showdown:
None of the six detectives were perfect, but they had different strengths:

• The "Super Sensitive" Detective (Singapore SIMs-B): This detective was great at not missing anyone. If a patient was actually intolerant, this detective almost always caught them (93% success rate). However, it also cried "Wolf!" a lot, flagging some people who were actually fine.
  • Best use: Use this one to screen patients. "Let's check everyone to make sure we don't miss a single case."
• The "Super Specific" Detective (Japan SAMT): This detective was great at being sure. If this detective said a patient was intolerant, they were almost certainly right (99% accuracy). But, it missed a lot of people who were actually intolerant.
  • Best use: Use this one to confirm a case. "Okay, we think this patient is intolerant. Let's run this specific test to be 100% sure before we stop their medication."

The Big Takeaway: Don't Let the Computer Decide Everything

The most important lesson from this paper is this: These digital detectives are helpful assistants, not the final boss.

You cannot just let a computer scan a file and say, "This patient is intolerant, stop the pill." That's too risky.

• The Computer's Job: To act as a decision-support aid. It says, "Hey Doctor, look at this patient. The rules suggest they might be having trouble. Please take a closer look."
• The Doctor's Job: To use clinical judgment. The doctor needs to talk to the patient, ask about their pain, check their history, and maybe try a different dose or a different pill.

Summary in a Nutshell

1. The Problem: It's hard to know exactly who can't take statin pills because the definition is fuzzy.
2. The Solution: We tried using computer programs (algorithms) to scan medical records to find these people.
3. The Finding: Only about 5% of people in this study were truly intolerant.
4. The Lesson: Different computer programs catch different types of cases. The best approach is to use one program to cast a wide net (find potential cases) and another to verify the catch (confirm the case), but always let a human doctor make the final call.

In short: Computers are great at finding clues, but humans are needed to solve the mystery.

Technical Summary

1. Problem Statement

Statin intolerance is a significant barrier to the effective primary prevention of cardiovascular diseases (CVD), affecting an estimated 5–15% of patients. However, the condition lacks a standardized taxonomy, with definitions varying widely across organizations (e.g., NLA, EAS, ILEP). This variability leads to inconsistent prevalence estimates and challenges in clinical management.

While Electronic Health Records (EHRs) contain vast amounts of data, identifying statin-intolerant patients automatically is difficult due to:

• Implicit or discrepant data: Symptoms and adverse events are often recorded unstructured or inconsistently.
• Nocebo effects: Patient expectations can mimic true physiological intolerance.
• Lack of validation: Existing electronic phenotyping algorithms (rule-based) have not been rigorously validated against a clinical reference standard in a general practice (GP) setting.

The study aims to identify statin-intolerant patients using EHR-detectable phenotyping algorithms, determine the prevalence of intolerance in a real-world Australian GP dataset, and validate these algorithms against a manually curated reference standard.

2. Methodology

The study utilized a four-stage methodology using the Electronic Practice Based Research Network (ePBRN) dataset, which covers approximately 249,345 patients across 11 GP sites in South-Western Sydney (2012–2019).

A. Dataset Selection and Preprocessing

• Cohort: Patients aged ≥18 prescribed statins for primary CVD prevention.
• Inclusion: Initiation of statins, at least 2 years of observation, and 3+ visits/prescriptions.
• Exclusion: History of CVD events (prior to or during observation), age >80, missing gender/birth year, or death during the period.
• Final Cohort: 15,583 patients (85,725 visits).

B. Phenotype Identification

Six rule-based electronic phenotyping algorithms were implemented using Python/SQL and R. These algorithms utilized structured data (diagnosis codes, medication records, lab values) based on definitions from various international bodies:

1. Minnesota Combined Rule-Based (CRB): NLA/EAS definitions.
2. Japan - Statin Induced Myopathy (SIMs): EAS definition (hospital/tertiary focus).
3. USA - SIMs: Custom definition (all settings).
4. Singapore - SIMs (A, B, C, D): CCWG definitions (tertiary focus).
5. Japan - Statin-Associated Muscle Toxicity (SAMT): Custom definition (trial-based).
6. NHS-UK Statin Intolerance Pathway: EAS/NICE guidelines.

C. Reference Standard Development

To validate the algorithms, a Reference Standard was created:

• Sample: A subset of 1,369 patients (potential cases identified by algorithms).
• Annotation: Two researchers manually reviewed records based on the European Atherosclerosis Society (EAS) definition (the most stringent clinical definition).
• Quality Control: Inter-Annotator Agreement (IAA) was calculated using Cohen's Kappa (0.894), and Deviation Scores (DS) were computed to ensure consistency.
• Gold Standard: The final consensus of the two annotators served as the "true" label for validation.

D. Validation Metrics

Algorithms were evaluated against the reference standard using:

• Accuracy, Sensitivity, Specificity.
• Positive Predictive Value (PPV) and Negative Predictive Value (NPV).
• F1 Score, Matthew's Correlation Coefficient (MCC), and Pearson's Correlation Coefficient (PCC).
• ROC and Precision-Recall (PR) AUC.

3. Key Results

A. Prevalence

• The overall prevalence of statin intolerance in the ePBRN dataset was 5.09% (794 out of 15,583 patients).
• This aligns with the lower end of the global range (5–15%), attributed to the use of the stringent EAS definition for the reference standard.
• Demographics: Intolerant patients were slightly older (mean 58.4 vs. 55.8 years), predominantly female (55.92%), and non-smokers (50.50%).

B. Algorithm Performance

No single algorithm performed optimally across all metrics, highlighting the trade-off between sensitivity (catching all cases) and specificity (avoiding false positives).

Algorithm	Best Performance Metric	Value	Interpretation
Singapore SIMs-B	Sensitivity	92.95%	Excellent at identifying true intolerant patients (low false negatives). High F1 (71.51%).
Singapore SIMs-B	Accuracy	57.05%	Highest overall accuracy among the group.
Japan SAMT	Specificity	99.13%	Excellent at identifying true tolerant patients (low false positives).
Japan SAMT	PPV	76.19%	Highest probability that a positive result is a true case.
Japan SAMT	Correlation (MCC/PCC)	0.05	Strongest agreement with the reference standard beyond chance.

• ROC AUC: All algorithms showed poor discriminating ability (AUC 0.50–0.52), indicating that simple rule-based approaches struggle to separate cases from controls in noisy EHR data without clinical nuance.
• Precision-Recall AUC: Japan SAMT performed best (0.593) in identifying true positives in this imbalanced dataset.

4. Key Contributions

1. Systematic Validation: This is one of the few studies to validate multiple international EHR phenotyping algorithms against a manually curated clinical reference standard in a primary care setting.
2. Algorithm Comparison: It demonstrates that no single "one-size-fits-all" algorithm exists.
   • Singapore SIMs-B is superior for screening (high sensitivity) to flag potential candidates for re-challenge or dose optimization.
   • Japan SAMT is superior for confirmation (high specificity/PPV) to justify discontinuing or switching therapy.
3. Prevalence Insight: It confirms that reported prevalence rates are highly dependent on the definition used and the setting (primary vs. tertiary care).
4. Clinical Workflow Proposal: The authors propose a two-stage diagnostic workflow:
   • Stage 1: Use Singapore SIMs-B to screen and flag potential intolerance.
   • Stage 2: Use Japan SAMT to confirm intolerance before making definitive treatment changes.

5. Significance and Limitations

Significance:

• The study supports the use of EHR phenotypes as decision-support aids rather than definitive diagnostic tools.
• It emphasizes that clinical judgment and patient engagement remain essential, particularly given the "nocebo" effect and the variability of muscle symptoms.
• It provides a framework for integrating genetic and phenotypic data to refine future predictive models.

Limitations:

• Reference Standard: The "gold standard" relied on manual annotation of routine clinical notes, which may contain documentation biases or missing data.
• Generalizability: The study was conducted in South-Western Sydney; results may not transfer directly to other healthcare systems or ethnic populations (CK thresholds vary by ethnicity).
• Prevalence Bias: The low prevalence (5.09%) impacts PPV/NPV calculations, potentially inflating the perceived utility of high-specificity algorithms.
• Ethnicity: The study did not adjust Creatine Kinase (CK) thresholds for ethnicity, which could introduce misclassification bias.

Conclusion

The paper concludes that while EHR-detectable phenotypes are valuable for identifying statin intolerance cohorts, they must be applied contextually. A hybrid approach using high-sensitivity algorithms for screening and high-specificity algorithms for confirmation, supported by clinical re-challenge and patient dialogue, offers the most robust strategy for managing statin intolerance in general practice.