📄 scientific communication and education

Estimating the replicability of Brazilian biomedical science

The Brazilian Reproducibility Initiative found that only 20–44% of 45 selected biomedical experiments were successfully replicated, revealing significantly lower effect sizes and higher variability in replication attempts while identifying protocol deviations, infrastructure challenges, and author publication rates as key factors limiting replicability in Brazil.

Original authors: Amaral, O. B., Carneiro, C. F. D., Neves, K., Sampaio, A. P. W., Gomes, B. V., Abreu, M. B. d., Tan, P. B., Mota, G. P. S., Goulart, R. N., Fernandes, N. R. d. S., Linhares, J. H., Ibelli, A. M. G., S

Published 2026-02-04

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Amaral, O. B., Carneiro, C. F. D., Neves, K., Sampaio, A. P. W., Gomes, B. V., Abreu, M. B. d., Tan, P. B., Mota, G. P. S., Goulart, R. N., Fernandes, N. R. d. S., Linhares, J. H., Ibelli, A. M. G., Sebollela, A., Andricopulo, A. D., Carvalho, A. A. V. d., Silva, A. P. e., Souza, A. S. O., Lima, A. C. C. d., Sousa, A. M. d., Birbrair, A., Borbely, A. U., Machado, A. M., Costa, A. d. C., Vasconcelos, A. P. d., Silva, A. H. B. d. L., Souza, A. d., Herrmann, A. P., Bastos, A. P. A., Nunes, A. C. C., Castrucci, A. M. d. L., Paula, A. B. R., Waltrick, A. P. F., Macedo, A. G. F. d., Mecawi, A. S., R

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine the world of scientific research as a massive, bustling kitchen where chefs (scientists) constantly publish new recipes (experiments) claiming to have discovered the secret to the perfect dish. For years, people have worried: "If I try to cook this recipe in my own kitchen, will I get the same delicious result, or will it be a total disaster?"

This paper is like a giant, organized "Taste Test" organized by the Brazilian Reproducibility Initiative. Instead of just trusting the original chefs, they invited 56 different kitchens (laboratories) across Brazil to try cooking 56 specific recipes that had been published before.

Here is how the experiment played out:

The Menu: They focused on three very common types of cooking techniques: checking cell health (MTT), copying genetic instructions (RT-PCR), and testing anxiety in mice (Elevated Plus Maze).
The Challenge: 143 different groups tried to recreate these dishes. However, just like in real life, some attempts were messy or didn't follow the rules, so an independent panel of judges threw out some of the results. They ended up with 90 valid attempts covering 45 original recipes.
The Verdict: The results were mixed. Depending on how strict the judges were, only 20% to 44% of the replications actually matched the original "perfect dish."
The Flavor Difference: When the replication kitchens did succeed, the results were often much weaker than the original. On average, the "flavor intensity" (the difference between groups) was 58% lower than what was originally claimed. Also, the results were much more "salty or sweet" depending on who was cooking (the results were 82% more variable).

Why did the recipes fail to replicate?

The paper found a few interesting patterns:

The "Easy" Dishes: The recipes that were originally considered the most difficult to cook were actually the ones that replicated best.
The Famous Chefs: If the original recipe was written by a chef who had published a lot of recipes recently, the replication was less likely to work. It's as if the famous chef was so busy they didn't write the instructions clearly enough.
The "Gut Feeling": Interestingly, the scientists trying to replicate the work were often pretty good at guessing whether their attempt would succeed or fail.
The Kitchen Chaos: A major reason for failure wasn't necessarily bad cooking skills, but the fact that the original instructions (protocols) were impossible to follow exactly. The replication kitchens often had to change the steps because their equipment was different, their mice were different, or their building's logistics didn't match the original setup.

The Bottom Line

This study doesn't say Brazilian science is "bad." Instead, it highlights that the "recipes" published in the past often relied on specific conditions that are hard to copy elsewhere. The paper suggests that to get better, more reliable results in the future, scientists need to write clearer instructions and acknowledge that their "kitchen" might not work exactly like everyone else's. It's a call to make science more transparent and robust, so that when a new recipe is published, anyone can cook it and get the same great taste.

Technical Summary: Estimating the Replicability of Brazilian Biomedical Science

Problem Statement
Despite growing global concerns regarding the replicability and reproducibility of published research, empirical data necessary to inform policy remain scarce. This gap is particularly acute in the context of Brazilian biomedical research, which has experienced rapid expansion over the last three decades yet lacks a systematic assessment of the reliability of its findings.

Methodology
To address this, the authors established the Brazilian Reproducibility Initiative, a multicenter replication project designed to test published experiments from Brazilian science. The initiative focused on three common experimental methods:

The MTT assay (cell viability).
Reverse transcription polymerase chain reaction (RT-PCR).
The elevated plus maze (EPM) (anxiety-related behavior in rodents).

The study involved 56 laboratories performing a total of 143 replications across 56 original experiments. An independent committee evaluated the quality of these replications, identifying 90 valid replications of 45 experiments for final analysis. The study utilized five predefined criteria to determine replication success and analyzed various statistical and contextual factors, including effect sizes, coefficients of variation, researcher predictions, and publication rates of original authors.

Key Results
The analysis yielded several critical findings regarding the state of replicability in the region:

Replication Rates: Success rates for the experiments varied between 20% and 44%, depending on the specific criteria applied.
Effect Size and Variability: In median terms, the ratios between group means in the replications were 58% lower than those reported in the original experiments. Conversely, the coefficients of variation in the replications were 82% higher.
Predictive Factors:
- Effect size decreases were less pronounced in MTT experiments, in original results exhibiting lower variability, and in studies originally considered more challenging to replicate.
- $t$ -values in the replications showed a positive correlation with researcher predictions regarding replicability.
- $t$ -values showed a negative correlation with the publication rate of the last author of the original articles.
Protocol Deviations: Deviations from preregistered protocols were frequent during the replication process. These were primarily attributed to factors inherent to the experimental models or issues related to infrastructure and logistics, rather than intentional protocol changes.

Significance and Claims
The paper concludes that its results highlight specific factors limiting the replicability of findings published by researchers in Brazil. By quantifying the discrepancy between original and replicated results and identifying the logistical and methodological hurdles involved, the study suggests pathways through which this scenario can be improved. The work serves as an empirical foundation for addressing reproducibility concerns within the Brazilian scientific community, aiming to foster a more robust research environment without overstating the universality of the findings beyond the specific context of the study.

Technical Summary: Estimating the Replicability of Brazilian Biomedical Science

More like this