📄 medicine

Exploratory reconstruction of the pulmonary embolism rule-out criteria and internal validation of a ridge- logistic model for database-recorded pulmonary embolism: a single-center retrospective cohort study

This single-center retrospective study found that while retrospectively reconstructed PERC criteria identified nearly all recorded pulmonary embolism cases, they lacked specificity for rule-out purposes, whereas an internally validated ridge-logistic model demonstrated significantly superior predictive performance for database-recorded outcomes, warranting further prospective multicenter evaluation before clinical adoption.

Original authors: Hua Fan, Xinbo Yu, Shengtao Yan, Yan Li, Rui Lian, Qingxia Liu, Yali Chen, Ping Xie, Xiaoyan Zhang, Jing Guo, Xiaoxia Song, Jiaqi Wang, Naicong Lu, Zhaoci Fu

Published 2026-08-07

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Hua Fan, Xinbo Yu, Shengtao Yan, Yan Li, Rui Lian, Qingxia Liu, Yali Chen, Ping Xie, Xiaoyan Zhang, Jing Guo, Xiaoxia Song, Jiaqi Wang, Naicong Lu, Zhaoci Fu

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery in a bustling city: a hidden, dangerous thief called a Pulmonary Embolism (PE). This thief is a blood clot that sneaks into the lungs, blocking the flow of air and potentially causing a fatal heart attack. The problem is that the thief wears a disguise; the symptoms—like chest pain, shortness of breath, or a racing heart—look exactly like the symptoms of a simple cold, a panic attack, or just being out of shape. If you check everyone in the city for the thief, you'll waste a massive amount of resources, expose people to unnecessary radiation from scans, and clog up the emergency rooms. But if you miss the thief, the consequences are deadly.

To solve this, doctors have a "rule-out" checklist called PERC (Pulmonary Embolism Rule-out Criteria). Think of PERC as a super-strict security guard at the city gate. The guard has eight specific questions: Are you over 50? Is your heart racing? Is your oxygen low? Do you have a swollen leg? If you answer "no" to all eight questions, the guard lets you walk right through without a full body scan. If you answer "yes" to even one, you get flagged for a closer look. This system works great when the guard is only checking people who already look very healthy. But what happens if you try to use this guard on a crowd of people who are already sick, tired, or just walking through the emergency room for other reasons? That's the big question this paper asks: Can we trust this old checklist when we look back at a messy pile of real-world hospital records, and can we build a better, smarter detective to replace it?

The Great Detective Experiment

In this study, a team of researchers from the China-Japan Friendship Hospital decided to put the PERC checklist to the test in a very specific way. They didn't go out and recruit new patients; instead, they dug through a digital time capsule: a database of 1,297 adult visits to their emergency department between June 2023 and May 2024. It was like looking at a giant photo album of people who had already been there, trying to figure out who actually had the PE thief and who didn't.

The Old Guard (PERC) vs. The Database
First, they tried to use the classic PERC checklist on these 1,297 people. The results were a bit of a mixed bag, mostly because the "crowd" they were testing wasn't the perfect, low-risk crowd the checklist was designed for.

The Good News: The checklist was incredibly good at catching the thief. It successfully identified 99.3% of the people who actually had a PE. It missed almost no one.
The Bad News: It was terrible at letting the innocent go. Because so many people in the emergency room were over 50 (one of the eight criteria), the checklist flagged almost everyone as "suspicious." In fact, it only cleared 9.7% of the people who didn't have a PE. It was like a security guard who stops everyone because they are all wearing shoes, even though only a few are actually thieves.

The researchers were careful to point out that because they couldn't verify the "innocent" people with follow-up scans (they only had the initial hospital notes), they couldn't say for sure if the few people the checklist did clear were truly safe. So, while the checklist caught nearly every thief, it wasn't a good tool for clearing the innocent in this specific, messy setting.

The New Detective (The Ridge-Logistic Model)
Since the old checklist was too blunt, the team decided to build a new, smarter detective using a method called "ridge-logistic regression." Imagine this as a super-powered calculator that doesn't just check a box for "yes" or "no." Instead, it looks at 13 different clues at once—like age, pulse, oxygen levels, coughing, chest tightness, and history of blood clots—and weighs them all together to give a precise probability score.

They trained this new detective on the same 1,297 visits and tested it rigorously using a technique called "cross-validation" (basically, splitting the data into five groups and testing the detective on one group while teaching it the other four, repeating this 20 times to make sure it wasn't just memorizing the answers).

The Results
The new detective was a clear winner in terms of accuracy:

The Score: The old PERC checklist had a "discrimination score" (called ROC AUC) of about 0.62. The new model scored a much higher 0.803. In the world of prediction models, moving from 0.6 to 0.8 is a huge leap; it means the new model is much better at telling the difference between a patient with a PE and one without.
The Comparison: The new model was significantly better than using the PERC score alone, or even adding a standard "illness severity score" (called NEWS2) to the PERC checklist. Adding NEWS2 actually made the PERC model slightly worse.
The Details: The new model found that clues like "previous blood clots," "coughing," "chest tightness," and "fainting" were the strongest indicators of a PE. Interestingly, after the model did its math, being male and having chest pain actually lowered the probability slightly, while being older increased it.

What This Means (and What It Doesn't)
The researchers are very clear about what they have and haven't proven. They have shown that this new mathematical model is better at predicting the recorded outcomes in their specific database. It's a more accurate "internal" tool.

However, they explicitly rule out the idea that this is ready for prime time in a real hospital.

No "Rule-Out" Safety: Because the study was retrospective (looking back) and didn't have long-term follow-up on the "negative" patients, they cannot claim that using this model would be safe for doctors to use right now to send people home.
Not a Final Solution: The model is a "proof of concept." It suggests that a complex, weighted model works better than a simple checklist in this specific dataset, but it needs to be tested in the real world, with real-time decisions and proper follow-up, before anyone can trust it to save lives.

The Bottom Line
This study is like a lab experiment where a new, high-tech scanner was built and found to be much sharper than the old, simple metal detector. The new scanner (the ridge model) sees the hidden threats much more clearly than the old checklist (PERC) ever could in this specific group of patients. But just because the scanner works in the lab doesn't mean we should start using it on the street yet. The authors are calling for a bigger, real-world trial to see if this new detective can actually keep the city safe before we hand it the keys to the emergency room.

Technical Summary: Exploratory Reconstruction of PERC and Internal Validation of a Ridge-Logistic Model for Pulmonary Embolism

Problem Statement
The Pulmonary Embolism Rule-out Criteria (PERC) are designed to identify very-low-risk patients in whom further diagnostic testing for pulmonary embolism (PE) may be unnecessary. However, PERC is intended for application only after a clinician has established a low pretest probability. Retrospective application of PERC in unselected emergency department (ED) populations is challenging because the necessary clinical gestalt (pretest probability assessment) cannot be reliably reconstructed from database records. Furthermore, retrospective databases often lack uniform verification of negative outcomes (e.g., via Computed Tomographic Pulmonary Angiography [CTPA] or longitudinal follow-up), making standard safety claims regarding rule-out strategies difficult to interpret. Additionally, while the National Early Warning Score 2 (NEWS2) standardizes the assessment of acute physiological derangement, its incremental value over PERC in PE prediction remains unclear.

Methodology
This study was a single-center, retrospective cohort analysis conducted at the China-Japan Friendship Hospital using an ED venous thromboembolism (VTE) database covering June 2023 to May 2024.

Cohort: The final analysis included 1,297 adult encounters (1,258 unique patients). The outcome was a binary PE label recorded in the database. All positive labels were CTPA-confirmed; negative labels were derived from recorded clinical diagnoses without longitudinal follow-up or uniform imaging verification.
PERC Reconstruction: The eight standard PERC criteria were reconstructed from electronic records. Undocumented symptoms were treated as absent.
Predictors: The study utilized 13 prespecified variables: age (continuous, per 10 years), male sex, pulse ≥100/min, oxygen saturation ≤94%, combined dyspnea/shortness of breath, cough, hemoptysis, chest pain, chest tightness, syncope, palpitation, unilateral leg swelling, and history of venous thromboembolism (VTE).
Models:
1. Ridge-Penalized Logistic Regression: A primary model using the 13 prespecified variables. The penalty parameter ( $C$ ) was tuned via nested cross-validation to minimize overfitting.
2. Comparators: Logistic models using PERC score alone, NEWS2 alone, and a combination of PERC score plus NEWS2.
Validation Strategy: Internal validation was performed using 20 repeats of five-fold stratified group cross-validation, ensuring all encounters from the same patient remained within a single fold to prevent data leakage. Performance metrics included ROC AUC, Precision-Recall (PR) AUC, Brier score, and calibration. Bootstrapping (2,000 replicates) was used for confidence intervals.

Key Results

PERC Performance: In the full cohort, reconstructed PERC demonstrated high sensitivity (99.3%) but very low specificity (9.7%), largely because the majority of patients were aged ≥50 years (automatically PERC-positive). Only one recorded PE case was PERC-negative. The Negative Predictive Value (NPV) was 99.1%, but the authors note this cannot be interpreted as evidence of rule-out safety due to the lack of low-pretest-probability selection and differential outcome verification.
Model Discrimination:
- The 13-feature ridge model achieved a median ROC AUC of 0.803 (2.5th–97.5th percentiles: 0.795–0.806) and a median PR AUC of 0.496.
- The PERC-score-only model achieved a median ROC AUC of 0.618.
- The NEWS2-only model performed poorly with a median ROC AUC of 0.550.
- Adding NEWS2 to the PERC score slightly reduced discrimination (AUC difference −0.015).
- The ridge model significantly outperformed the PERC-score model, with an AUC difference of 0.216 (95% CI 0.156 to 0.274).
Calibration: The ridge model showed good calibration with an intercept of 0.002 and a slope of 1.004 on averaged out-of-fold predictions.
Subgroup Analysis: The ridge model maintained discrimination in the age <50 subgroup (AUC 0.892) and the age ≥50 subgroup (AUC 0.795), though confidence intervals were wider in the younger cohort.

Significance and Claims
The paper claims that while retrospectively reconstructed PERC successfully identified nearly all recorded PE cases in this unselected database, its low specificity and the study's design limitations (lack of uniform negative verification and pretest probability assessment) preclude any claim of clinical rule-out safety.

The primary contribution is the development and internal validation of an interpretable ridge-logistic model that significantly improves the discrimination of the database-recorded PE outcome compared to PERC-based or NEWS2-based approaches. The authors emphasize that this model is designed for probability estimation rather than binary diagnosis. They explicitly state that the model should not be deployed clinically at this stage. Instead, the findings support the need for prospective, multicenter evaluation with predefined patient selection (low pretest probability) and uniform outcome ascertainment before any clinical implementation can be considered. The study serves as a descriptive stress test of reconstructed PERC and a proof-of-concept for a penalized prediction model using routinely available ED variables.

The Great Detective Experiment

Technical Summary: Exploratory Reconstruction of PERC and Internal Validation of a Ridge-Logistic Model for Pulmonary Embolism

More like this