AI-Driven Paraphrasing and Evidence Recycling in Global Health Meta-Analyses: Introducing the EVIDENCE Framework for Detection and Prevention

Technical Summary: The EVIDENCE Framework for Detecting AI-Driven Evidence Recycling

Problem Definition: Evidence Recycling
The paper identifies a novel, scalable threat to research integrity termed "evidence recycling." Unlike traditional plagiarism (verbatim copying) or duplication bias (overlapping study populations), evidence recycling involves the AI-driven semantic paraphrasing of existing research findings. Large Language Models (LLMs) rewrite core conclusions and effect estimates from a single primary study into multiple, linguistically distinct "new" studies. Because the text is semantically rewritten rather than verbatim, it bypasses standard plagiarism detection tools.

The authors argue that this creates an illusion of novel evidence without new primary data. In meta-analyses, this leads to the treatment of non-independent effect sizes as independent observations, artificially reducing variance and inflating Type I error rates (false positives). This threat is particularly acute in Low- and Middle-Income Countries (LMICs), where verification capacity is limited, and in global health policy, where such errors could misdirect scarce resources. Current guidelines, such as PRISMA AI and TRIPOD AI, address transparency in AI usage but lack operational mechanisms to verify that every effect size in a meta-analysis originates from a unique primary study.

Methodology
This paper reports a conceptual framework development supported by a structured narrative synthesis and a proof-of-concept pilot. The methodology followed four distinct steps:

1. Literature Search: A search of PubMed, arXiv, and Retraction Watch (January 2020 – March 2025) identified 47 records. Ten primary empirical sources were selected to document AI-related citation fabrication, retraction propagation, and automation bias.
2. Framework Development: Failure modes identified in the literature were mapped to potential editorial safeguards. These were grouped into eight domains, forming the acronym EVIDENCE. The framework was iteratively refined to ensure mutual exclusivity and collective exhaustiveness.
3. Face-Validity Assessment: Two independent methodological experts (one in systematic review methods, one in AI ethics) assessed the draft framework. Their feedback led to structural changes, including the addition of a "Diversity of Data" domain. The experts rated the framework as clear and feasible (mean scores of 4.5/5 for clarity and 4.0/5 for feasibility).
4. Proof-of-Concept Pilot: The operational core of the framework, the "dependency matrix," was applied to three recently published open-access global health meta-analyses (2023–2024). A reviewer attempted to trace every extracted effect size to a unique DOI or PMID within 30 minutes per review to simulate an editorial spot-check.

Key Contributions: The EVIDENCE Framework
The primary contribution is the EVIDENCE Framework, an eight-domain methodological checklist designed to counter evidence recycling. The framework extends current guidance by introducing five new domains not covered by existing guidelines like PRISMA AI:

• E (Evaluation of originality): Verifying that cited sources are real publications via DOI/PMID lookup.
• V (Verification of sources): The operational core requiring a dependency matrix—a structured table mapping every extracted effect size to a unique source DOI or PMID.
• I (Integrity in synthesis): Ensuring statistical accounting for non-independent effect sizes (e.g., using Robust Variance Estimation).
• D (Diversity of data): Checking for distinct patient populations, settings, and time periods to prevent "paraphrased diversity."
• E (Ethical AI use): Mandating disclosure of AI stages and human verification of outputs.
• N (Novel contribution): Requiring explicit statements that no included study is a paraphrased version of another.
• C (Contextual relevance): Assessing the appropriateness of conclusions for specific settings (especially LMICs) and proposing a "Two Reviewer Rule" for resource-constrained settings (a 20% random sample trace-back).
• E (Editorial responsibility): Directing journals to mandate the checklist and dependency matrix before peer review.

Results

• Framework Structure: The eight-domain framework was successfully developed and validated for face validity.
• Pilot Application: When applied to three published meta-analyses, the dependency matrix revealed source-tracing gaps in all three. In one specific meta-analysis, two of twelve effect sizes could not be traced to a unique DOI within the allotted time, generating flags for further audit.
• Feasibility: The pilot demonstrated that constructing a dependency matrix is feasible (taking 18–26 minutes per review in the pilot) and capable of identifying potential tracing failures.
• Resource Constraints: The paper proposes the "Two Reviewer Rule" (2R-AI) as a lightweight alternative for LMICs, requiring only a 20% random sample trace-back, estimated to take two additional hours per review.

Significance and Claims
The authors position the EVIDENCE Framework as a practical, low-cost, and context-sensitive tool to address an emergent threat that current editorial safeguards cannot detect. They explicitly state that evidence recycling is not a hypothetical risk but a scalable reality enabled by generative AI.

The paper's significance lies in shifting from abstract calls for "human oversight" to providing a concrete, immediately implementable tool: the dependency matrix. The authors claim that five of the eight domains address critical gaps in current literature. They do not claim to have quantified the prevalence of evidence recycling globally, noting that their pilot was not powered for prevalence estimation. Instead, they invite five major journals to conduct a six-month field test to validate the framework's real-world effectiveness and quantify the prevalence of the issue. The framework is presented as an actionable starting point to prevent a potential crisis in global health research integrity.