Obvious Objectivity of Quantification and the Replication Crisis: the exemple of Human Resource Management Research

In the world of business management, there is a strong belief that numbers tell the truth. When a company wants to understand its employees, it often turns to surveys where people rate their feelings on a scale from one to five, or from "strongly disagree" to "strongly agree." These ratings are then fed into computer programs that crunch the numbers to find patterns, test ideas, and declare what is true about how people work. This approach, known as quantitative research, has become the standard way to study human resources. It promises a level of objectivity and precision that feels scientific, much like measuring the temperature or the weight of an object. However, a growing number of scientists are worried that this confidence in numbers might be misplaced. They argue that the tools used to measure human thoughts and behaviors are often treated as if they were measuring physical things, even though feelings and attitudes do not behave like temperature or weight. This tension has led to a crisis of confidence, where studies that seem solid on paper fail to produce the same results when others try to repeat them.

Two researchers at the Université du Québec à Rimouski decided to investigate whether this crisis is happening inside the field of human resource management. They asked a simple but profound question: are the methods researchers use to study employees actually as rigorous as they appear, or are they just following a set of habits that create an illusion of certainty? To find out, they did not run a new experiment or collect fresh data from a single company. Instead, they performed a massive review of existing research. They gathered 398 articles published in the top ten international journals dedicated to human resource management. From this large pile, they filtered down to 82 studies that used numbers and statistics to answer questions about how people are managed, how they lead, and how they behave in organizations. Their goal was to see how these studies handled three specific, common practices: how they treated survey ratings, how they tested their ideas, and who they asked for answers.

The researchers found that the field is deeply entrenched in a system that prioritizes the appearance of rigor over actual methodological soundness. First, they looked at how researchers handle survey ratings. In these studies, people are often asked to choose from a list of options like "strongly disagree," "disagree," "neutral," "agree," and "strongly agree." These are called ordinal scales because they show an order, but the distance between "disagree" and "neutral" is not necessarily the same as the distance between "neutral" and "agree." Despite this, the vast majority of the studies treated these ratings as if they were precise measurements, like inches or kilograms. They calculated averages and used complex statistical formulas that assume equal spacing between the choices. The researchers found that 66 percent of the quantitative studies did this, effectively turning subjective opinions into fake numbers that look precise but lack a real mathematical foundation.

Next, the team examined how these studies tested their ideas. Most of the research relied on a standard procedure called null hypothesis significance testing. In simple terms, this is a method where researchers try to prove that a result is not just a random fluke. If the math shows the result is unlikely to happen by chance, they declare it a success. The researchers found that 93 percent of the studies used this method as their primary way of deciding what was true. They noted that this has become a routine habit rather than a thoughtful choice. The studies rarely questioned whether this method was appropriate for the specific question being asked. Instead, the presence of a specific statistical result became a signal that the work was scientific, regardless of whether the underlying data supported such a strong conclusion.

The third issue the researchers uncovered concerned who was actually being studied. To make a valid claim about a whole group of people, a researcher ideally needs to pick participants randomly, like drawing names from a hat, so that everyone has an equal chance of being included. This ensures the results can be applied to the wider population. However, the review showed that 98 percent of the studies used non-random samples. This means the researchers simply asked whoever was available, such as employees at a specific company that agreed to participate, or people who volunteered to fill out a survey online. Because these groups are not random, they often share specific traits that skew the results. For example, people who volunteer to answer a survey about job satisfaction might be happier or more engaged than the average worker. The researchers found that almost none of the studies admitted this was a major problem. Instead, they presented their findings as if they applied to all workers, ignoring the fact that their data came from a very specific, non-random group.

Perhaps the most striking finding was not just that these practices were common, but that they were rarely discussed as problems. When researchers write a paper, they usually include a section at the end to list the study's weaknesses or limitations. The authors of this review checked these sections to see if the researchers admitted to using non-random samples, treating ratings as precise numbers, or relying heavily on standard statistical tests. They found that almost no one did. Only one article out of the 82 mentioned the overuse of statistical testing as a limitation. The other two major issues were almost completely ignored. This silence suggests that these methods have become so normalized that researchers no longer see them as choices that need justification. They are simply the way things are done.

The researchers argue that this situation creates a self-reinforcing cycle. Because everyone uses these methods, they look legitimate. Because they look legitimate, journals publish them, and students are taught to use them. This creates a form of "hegemonic robustness," where the field feels strong and scientific because it follows a set of shared rules, even if those rules are built on shaky ground. The result is a body of knowledge that looks precise but may not be reliable. When other scientists try to repeat these studies with different groups of people or slightly different methods, the results often do not match. This is the heart of the replication crisis: the inability to reproduce findings that were once considered solid.

The paper concludes that the solution is not to stop using numbers or to abandon the study of human resources. Instead, the researchers call for a more honest and reflective approach. They suggest that scientists need to be transparent about the limits of their tools. If a survey rating is not a precise measurement, they should say so. If a sample is not random, they should explain how that limits what they can claim. They urge the field to move away from treating statistical formulas as magic spells that guarantee truth and toward a deeper understanding of what the data actually represents. By acknowledging the imperfections in their methods, researchers could build a more honest and durable foundation for understanding how people work, rather than relying on a system that produces the appearance of certainty while hiding its own flaws.