Regulatory effect of document samples on the fluctuation of journal impact factor
This study demonstrates that recalculating journal impact factors by excluding extreme citation outliers from the document sample, with an optimal retention threshold of 90%, effectively reduces indicator fluctuations and enhances the stability and rationality of journal evaluation systems.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are trying to judge the quality of a restaurant based on its average customer rating. Usually, you'd take every single review, add them up, and divide by the number of reviews. That gives you the "Impact Factor" (or in this case, the Journal Impact Factor) for a scientific magazine.
However, there's a problem with this simple math. If one person writes a review saying, "This is the best meal in the history of the universe!" and gives it 100 stars, while another says, "The food was okay," and gives it 3 stars, that one 100-star review can skew the entire average. Suddenly, a restaurant that usually serves decent food looks like a five-star masterpiece, even if the next day it goes back to serving average meals.
This is exactly what happens with scientific journals. A single, incredibly popular paper (a "viral hit") can make a journal's score jump wildly one year and crash the next, making it hard to tell if the journal is actually good or just lucky.
The Problem: The "Outlier" Effect
The authors of this paper looked at nine famous scientific journals (including giants like Nature, Science, and Cell, plus several physics journals). They noticed that these journals often have wild swings in their scores.
Sometimes, a journal publishes a paper that gets cited (mentioned) thousands of times by other scientists. This is great, but because the current scoring system counts every paper equally, that one superstar paper drags the average up so high that it doesn't represent the quality of the other 99 papers the journal published that year. It's like a classroom where one genius student gets a perfect score, and suddenly the whole class looks like geniuses, even if the rest of the students are struggling.
The Solution: The "Trimming" Strategy
To fix this, the researchers proposed a new way to calculate the score. They called it a "dual-end trimming" strategy.
Think of it like this: Imagine you have a long line of students ranked by their test scores.
- The Old Way: You take the average of everyone's score.
- The New Way: You look at the line, and you quietly ask the top 1% of students (the superstars) and the bottom 1% of students (the ones who might have had a bad day) to step aside. Then, you calculate the average of the remaining 98% of students.
By removing the "extremes" at both ends of the line, you get a score that represents what the typical student (or paper) is actually doing, rather than being swayed by the outliers.
What They Found
The researchers tested this idea on their nine journals over 16 years. Here is what happened:
- Stability: When they removed the top and bottom papers, the wild swings in the scores disappeared. The scores became much smoother and more predictable.
- The "Sweet Spot": They tried removing different amounts of papers (1%, 2%, 5%, etc.). They found that removing just a tiny bit (the top and bottom 1% each, leaving 98%) helped a lot, but removing a bit more helped even more.
- The Magic Number: They determined that keeping 90% of the papers (removing the top 5% and bottom 5%) was the perfect balance.
- If you keep 100%, the scores are too wobbly.
- If you keep only 50%, you are throwing away too much information, and the score might not represent the journal well anymore.
- 90% is the "Goldilocks" zone: it removes the crazy outliers that cause the spikes, but keeps enough data to show the journal's true, steady quality.
The Takeaway
The paper concludes that by using this "trimming" method, we can stop scientific journals from having their scores hijacked by a single lucky hit. It makes the evaluation system fairer and more honest. Instead of a rollercoaster ride of scores, we get a steady, reliable measure of how good a journal really is, paper by paper.
In short: Don't let one superstar paper fool you. Look at the middle 90% to see the real story.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.