Comparing Existing Algorithms for Retrieving Pregnancy-related Adverse Event Reports

This study compares three rule-based algorithms designed to retrieve pregnancy-related adverse event reports from pharmacovigilance databases, revealing that differences in their flagged reports stem primarily from varying scopes regarding age restrictions, normal pregnancies, and paternal exposure, thereby highlighting the need for professionals to select the most appropriate tool based on their specific research needs.

The Gist

Imagine you are a detective trying to find specific clues in a massive, chaotic library filled with millions of books. Your mission? To find every single book that mentions a specific topic: pregnancy.

In the world of medicine, this "library" is a database of safety reports about drugs. When people get sick after taking medicine, doctors write reports. But here's the problem: there is no single, standard "tag" or "sticker" on these reports that says, "This happened during a pregnancy." It's like trying to find all the books about "apples" in a library where some people wrote "fruit," others wrote "red things," and some just wrote "snack."

To solve this, three different teams of detectives (researchers) built three different search robots (algorithms) to scan these databases and find the pregnancy-related reports. Each robot was built for a different library (FAERS, EudraVigilance, and VigiBase) and had its own unique set of rules.

The Three Robots and Their Rules

The paper compares these three robots to see how well they work and what they catch.

1. Robot A (The "Wide Net" Detective): This robot casts a very wide net. It doesn't care about the age of the person. It will grab reports from anyone who might be pregnant, even if the report is a bit vague.

   • Analogy: Imagine a fisherman using a giant net that catches everything, from tiny fish to big whales, just to make sure he doesn't miss a single one.

2. Robot B (The "Strict Filter" Detective): This robot is very picky. It has a rule that says, "If it's a normal, healthy pregnancy, ignore it. If it's just a failed attempt at birth control, ignore it." It only wants the reports where something actually went wrong.

   • Analogy: This is like a bouncer at a club who only lets in people with a specific VIP pass and turns away everyone else, even if they are just standing outside.

3. Robot C (The "Mother-Only" Detective): This robot is focused strictly on the mother. It ignores reports where the father took the medicine, even if the father's exposure might affect the pregnancy.

   • Analogy: This is like a security guard who only checks the woman's ID at the door and doesn't even look at the man standing next to her.

The Great Comparison

The researchers took these three robots and ran them through two huge libraries (VigiBase and FAERS). Here is what they found:

• The Numbers: Robot C found the most reports (like catching the most fish), while Robot B found the fewest.
• The Differences:
  • When Robot A found reports the others missed, it was usually because it was looking at older people or didn't have strict age limits.
  • When Robot B missed reports, it was because it was ignoring "normal" pregnancies or failed contraception attempts.
  • When Robot C missed reports, it was simply because it ignored cases where the father took the drug.

The Big Takeaway

The main lesson of this paper is that there is no single "perfect" robot.

If you are a safety expert trying to study a drug, you need to know which robot to use.

• If you want to catch every possible pregnancy case, even the weird ones, you might want Robot A.
• If you only care about serious complications and want to filter out the "noise," Robot B is better.
• If you are specifically studying the direct impact on the mother, Robot C is your tool.

In simple terms: Just like you wouldn't use a butterfly net to catch a shark, you shouldn't use the wrong algorithm to find pregnancy safety data. Understanding the strengths and weaknesses of each tool helps doctors and researchers pick the right one for their specific job, ensuring they get the clearest picture of how safe medicines are for pregnant people.

Technical Summary

1. Problem Statement

Post-marketing surveillance is critical for establishing the safety profiles of medicinal products, particularly for populations often excluded from clinical trials, such as pregnant individuals. However, a significant barrier exists in pharmacovigilance: the lack of a standardized pregnancy indicator in electronic adverse event (AE) reports. This absence makes it difficult to accurately identify and retrieve reports related to pregnancy exposures within large databases like FAERS (FDA Adverse Event Reporting System), EudraVigilance, and VigiBase. Consequently, studying safety concerns specific to pregnancy is complicated. While three distinct rule-based algorithms (Algorithm A for FAERS, B for EudraVigilance, and C for VigiBase) were recently developed to address this, their differences in scope and design had not been systematically compared.

2. Methodology

The study was a collaborative effort among the original authors of the three algorithms. The methodology involved:

• Harmonization: The rules governing the three algorithms were standardized to allow for direct comparison.
• Implementation: The harmonized rules were implemented into a unified R package. This tool enabled the execution of all three algorithms on two major global databases: VigiBase (WHO) and FAERS (FDA).
• Analysis: The researchers analyzed the reports flagged by each algorithm, focusing on key characteristics and the volume of reports retrieved. They specifically investigated reports that were exclusively flagged by one algorithm to understand the impact of specific rule variations.

3. Key Contributions

• Unified Framework: The creation of an R package that allows the simultaneous execution and comparison of three distinct pregnancy retrieval algorithms across different databases.
• Cross-Database Validation: The study successfully applied algorithms originally designed for specific databases (e.g., Algorithm A for FAERS) to others (e.g., VigiBase), demonstrating their adaptability.
• Granular Comparison: Instead of treating the algorithms as black boxes, the study dissected the specific logical rules (e.g., age restrictions, exclusion of normal pregnancy, paternal exposure) that drove differences in output.

4. Results

The study quantified the retrieval performance and identified the sources of divergence:

• Retrieval Volumes:
  • In VigiBase: Algorithm A flagged 235,653 reports; Algorithm B flagged 279,515; and Algorithm C flagged 446,957.
  • In FAERS: Algorithm A flagged 265,015; Algorithm B flagged 260,734; and Algorithm C flagged 350,479.
• Exclusive Retrievals (Unique to each algorithm):
  • VigiBase: Algorithm A uniquely retrieved 994 reports; Algorithm B retrieved 3,248; Algorithm C retrieved a significant 142,324.
  • FAERS: Algorithm A uniquely retrieved 1,528; Algorithm B retrieved 1,100; Algorithm C retrieved 59,643.
• Root Causes of Divergence: The analysis revealed that the massive discrepancies (particularly the high number of unique reports for Algorithm C) were driven by specific scope definitions:
  • Algorithm A: Included reports with no age restriction, capturing a broader demographic.
  • Algorithm B: Explicitly excluded reports involving "normal pregnancy" and "ineffective contraception," narrowing its scope to adverse outcomes.
  • Algorithm C: Excluded paternal exposure reports, which significantly reduced its count compared to the others in some contexts, though its overall volume was highest, suggesting other inclusive criteria.

5. Significance

The study concludes that the differences in flagged reports are primarily due to varying scopes and implementation rules rather than technical errors. This has several critical implications for the field of pharmacovigilance:

• Algorithm Selection: Professionals must understand the specific inclusion/exclusion criteria of each algorithm to select the one that best fits their research question (e.g., whether they need to include paternal exposures or normal pregnancy outcomes).
• Standardization Awareness: The findings highlight the urgent need for standardized pregnancy indicators in electronic reporting to reduce reliance on complex, divergent rule-based algorithms.
• Research Reproducibility: By clarifying the overlaps and gaps between these tools, the study empowers researchers to interpret safety data more accurately and ensures that pregnancy-specific safety signals are not missed or misinterpreted due to algorithmic bias.