State of play in individual participant data meta-analyses of randomised trials: Systematic review and consensus-based recommendations

This systematic review and consensus study evaluates the current landscape and methodological practices of 605 individual participant data (IPD) meta-analyses of randomised trials, identifying persistent shortcomings in transparency and data utilization while providing expert recommendations to enhance future research quality and clinical decision-making.

The Gist

Imagine that medical research is like a massive library where thousands of individual studies are stored on separate shelves. Usually, when doctors want to know the best treatment for a patient, they look at the "summary cards" (aggregate data) that researchers have written about these studies. These summary cards tell you the average result, but they often miss the tiny details about who specifically benefited or struggled.

Individual Participant Data (IPD) Meta-Analysis is like asking the library to pull out every single original notebook from every study, put them all on one giant table, and read every single page together. This allows researchers to see the full picture, not just the summary.

This paper is a "state of the union" report on how well researchers are currently doing this "giant table" exercise. The authors, a team of experts from around the world, did three main things to figure out the current situation:

1. The Big Picture Scan (The Landscape)

The team looked at 605 of these "giant table" studies published between 1991 and 2024.

• The Trend: Think of this like a popular TV show. For a long time, the number of these studies grew every year, reaching a peak in 2019. But since then, the number has leveled off (plateaued) at about 60 new studies a year. It's not growing anymore; it's just staying steady.
• The Topics: Most of these studies focus on heart disease and cancer. These are the "blockbuster movies" of the medical world. Other areas, like mental health, are represented but less frequently.
• The Location: Most of these studies are led by researchers in Europe, Australia, and the US. It's like the library is mostly staffed by people from a few specific countries, leaving other parts of the world underrepresented.

2. The Deep Dive (The Recent Check-Up)

The team then took a closer look at 100 of the most recent studies (published between 2022 and 2024) to see how they were actually being done. They found a mix of good news and bad news:

The Good News (Things are getting better):

• Better Checklists: Most researchers are now using a standard checklist called PRISMA-IPD to make sure they don't miss steps, much like a pilot using a pre-flight checklist.
• Quality Control: Most studies now check if the original experiments were done correctly (risk of bias), whereas in the past, they often skipped this.
• Fairer Comparisons: Researchers are getting better at comparing patients fairly, ensuring they aren't mixing up "within-study" differences with "between-study" differences (a common mistake that used to skew results).

The Bad News (Where they are still stumbling):

• Secret Recipes: Only about one-third of the studies published their "recipe" (a protocol) or their "cooking plan" (statistical analysis plan) before they started cooking. Without this, it's hard to know if they changed the recipe halfway through to get a better result.
• Missing Ingredients: They often fail to look for "unpublished" studies (trials that were done but never written up). This is like trying to bake a cake but ignoring the ingredients sitting in the pantry that no one has written about yet.
• Wasted Potential: Even though they have all the raw data, they rarely use it to check if the original data was trustworthy or to fix missing information. It's like having a high-definition camera but taking blurry photos because you didn't adjust the focus.
• Bureaucracy: Getting the data from the original researchers is often slow and full of red tape. The experts say the process is like trying to borrow a book from a library that requires three forms, a fingerprint, and a week-long wait.

3. The Expert Roundtable (The Consensus)

The authors gathered 24 top experts (statisticians, doctors, and researchers) for a workshop to agree on how to fix these problems. They came up with a set of recommendations, which can be summarized as:

• For the Authors (The Cooks):

  • Be Transparent: Publish your recipe and plan before you start.
  • Share the Code: If you used a computer program to crunch the numbers, share the code so others can check your math.
  • Look Everywhere: Don't just look at published papers; search for unpublished trials and try to get their data too.
  • Use the Data: Use the raw data to check for errors and missing pieces, don't just ignore them.

• For the Teachers and Methodologists (The Librarians):

  • Build a Toolbox: Create an easy-to-use online "toolbox" with templates and examples so researchers don't have to reinvent the wheel.
  • Train More People: Teach more researchers, especially in countries where this work is rare, how to do it properly.
  • Update the Rules: The current rulebook (PRISMA-IPD) is a bit old. It needs an update to include modern requirements like checking for data integrity and handling missing data.

The Bottom Line

The paper concludes that while the "giant table" method (IPD meta-analysis) is a powerful tool for finding the best medical treatments, it is currently being underused and sometimes done poorly. The experts believe that if researchers become more transparent, share their data and plans more openly, and if the process of getting data becomes less bureaucratic, we can generate much higher quality evidence to help doctors and patients make better decisions.

In short: We have the tools to build a better map of medical evidence, but we need to stop hiding our blueprints, start looking for all the missing pieces, and make it easier for everyone to join the construction crew.

Technical Summary

Technical Summary: State of play in individual participant data meta-analyses of randomised trials

Problem Statement
Individual Participant Data (IPD) meta-analyses are considered the "gold standard" for synthesizing evidence from randomised trials, offering superior data availability and the ability to conduct more robust, personalized analyses compared to aggregate data meta-analyses. However, despite their potential to inform clinical practice and policy, historical reviews indicate that IPD meta-analyses have frequently failed to adhere to best practices in design, conduct, analysis, and reporting. Previous methodological assessments are now over a decade old and predate significant developments in the field, such as the introduction of the PRISMA-IPD reporting guideline, new data sharing policies, and advanced methods for assessing trial integrity. Consequently, there is a lack of up-to-date, comprehensive evidence regarding the current landscape of IPD meta-analyses and the specific methodological gaps that persist.

Methodology
This study employed a three-part design to evaluate the current state of IPD meta-analyses and derive consensus recommendations:

1. Systematic Review (Part 1): A comprehensive search of MEDLINE, Embase, and the Cochrane Database of Systematic Reviews was conducted to identify all published IPD meta-analyses of randomised trials evaluating intervention effects up to February 2024. This included 605 studies (282 newly identified and 323 from a previous review up to 2019).
2. In-Depth Review (Part 2): A focused analysis of 100 pairwise IPD meta-analyses published between February 2022 and February 2024. This subset was examined for detailed data on retrieval rates, reporting standards, risk of bias assessments, and statistical analysis strategies. Data extraction followed PRISMA-IPD items and previous methodological projects, with initial duplicate extraction followed by single extraction once reviewer agreement was near-perfect.
3. Consensus Process (Part 3): An adapted nominal group technique was used with a panel of 24 international experts (methodologists, statisticians, and experienced IPD authors). The panel reviewed findings from Parts 1 and 2 across four workshops to interpret results and generate consensus-based recommendations for authors, educators, and methodologists.

Key Contributions and Results

• Landscape and Trends: The number of IPD meta-analyses published annually increased over time, peaking at 73 in 2019, before plateauing at approximately 60 per year. The majority of studies focus on cardiovascular disease (19%) and cancer (18%), with 70% of corresponding authors based in Europe. While PROSPERO registration has increased (65% for studies published 2021–2024), the proportion of IPD meta-analyses published as Cochrane reviews has declined significantly (from 16% pre-2015 to 3% in 2021–2024).
• Current Practices and Shortcomings:
  • Transparency: Only 33% of recent studies provided a published protocol, and only 29% provided a separate statistical analysis plan (SAP). Analysis code was rarely made available.
  • Data Retrieval: The median trial IPD retrieval rate was 85.7%, and the median participant retrieval rate was 99.2%. However, only 15% of studies obtained IPD from unpublished trials, and clinical trial register searches were reported in only 55% of studies.
  • Methodological Rigor: While 82% of studies reported risk of bias assessments, only 7% reported using the IPD itself to inform these assessments. Only 1% reported an estimand, and 4% described a sample size or power calculation.
  • Analysis: One-stage analysis was the most common modeling approach (52%). While 85% of studies conducted participant-level subgroup analyses, only 41% correctly separated within- and across-trial information to avoid aggregation bias.
  • Reporting: Although 68% cited PRISMA-IPD, reporting often lacked detail on how guidelines were followed, particularly regarding data checking processes and trustworthiness assessments.

Significance and Recommendations
The study concludes that while the conduct and reporting of IPD meta-analyses have improved in recent years—particularly in the adoption of reporting guidelines and risk of bias assessments—significant shortcomings remain. These include insufficient pre-specification of methods, sub-standard transparency, and a failure to fully leverage the unique advantages of IPD (e.g., addressing missing data, assessing trial integrity, and including unpublished evidence).

To address these gaps, the consensus panel provided specific recommendations:

• For Authors: Prioritize transparency through prospective registration, publication of detailed protocols and SAPs, and sharing of analysis code. Authors should maximize the value of IPD by searching trial registries, seeking data from unpublished trials, using IPD to inform risk of bias and trustworthiness assessments, and appropriately separating within- and across-trial information in subgroup analyses.
• For Methodologists and Educators: Develop practical resources, such as an open-access "IPD toolbox" containing templates and worked examples, to facilitate the application of best practices. There is a need for targeted training and capacity building, particularly in underrepresented geographical and clinical areas.
• For Future Development: The panel highlighted the need to update the PRISMA-IPD guideline to reflect current methodological expectations (e.g., trustworthiness checks) and to streamline data sharing pathways to reduce bureaucratic barriers.

The authors assert that implementing these consensus-based recommendations is essential to ensure future IPD meta-analyses generate robust, high-quality evidence capable of effectively informing clinical decision-making and policy in an era of personalized medicine.