The Why, What, How and When of Big Science evaluation. Perspectives on socio-economic assessment of research infrastructures

Large-scale scientific facilities, often called "big science," are the massive engines of modern discovery. Think of them as the giant telescopes peering into the distant universe, the particle accelerators smashing atoms to reveal the building blocks of matter, or the fusion reactors attempting to replicate the power of stars. These are not just laboratories; they are research infrastructures, colossal machines that cost billions to build and run, requiring decades of planning and international cooperation. Because they are so expensive and rely on public money, a critical question has emerged: do they actually pay off for society? It is not enough to know that they produce new scientific papers. Governments and citizens want to know if these investments spark new industries, create jobs, improve health, or solve global problems. This is the realm of socio-economic evaluation, a field dedicated to measuring the real-world value of these scientific giants. For years, the expectation has been that every major facility would rigorously prove its worth, but the reality on the ground has been far more complicated.

A new analysis brings together experts from across the European scientific landscape to examine why this gap exists and how to fix it. The authors, drawing from a structured discussion among practitioners and policymakers, argue that the current way we judge these facilities is broken. Instead of treating evaluation as a one-time box-checking exercise done only when a funder demands it, they propose that it must become a continuous, integrated part of the facility's life. The paper suggests that we need to stop trying to prove that a specific machine caused a specific invention, a task that is often impossible, and instead focus on how the facility contributed to a broader ecosystem of innovation. The central finding is that for these evaluations to be useful, they must be planned from the very first day a project is conceived, supported by dedicated data systems, and communicated with honesty about what we know and what remains uncertain.

The reasons for evaluating these projects are threefold, and each requires a different approach. First, there is the economic argument. Science is often a public good, meaning the benefits of discovery spill over to everyone, not just the person who paid for it. Private companies rarely invest enough in basic research because they cannot capture all the profits. Public funding fills this gap, but it must be justified by showing that the benefits outweigh the costs. Second, there is the issue of legitimacy. As public scrutiny grows, scientists must be able to explain to ordinary citizens why spending billions on a particle accelerator is a good idea. This is not just about public relations; it is about maintaining the social contract that allows these projects to exist. Third, there is a practical need. When designing a massive facility, there are often different options for how to build it or when to build it. Evaluation can help decision-makers choose the path that offers the most value, but this only works if the analysis happens before the blueprints are finalized and the money is spent.

However, measuring this value is incredibly difficult. One major hurdle is the problem of attribution. In the complex web of modern innovation, it is nearly impossible to draw a straight line from a specific investment in a research facility to a specific product on a store shelf. Consider the history of the Global Positioning System, or GPS. It began with a simple observation of a satellite signal in the 1950s, leading to a navigation system for submarines, and eventually to the technology that guides billions of smartphones today. The commercial applications were not visible at the start; they emerged decades later through a long, winding process of discovery. Trying to claim that the original research "caused" the smartphone industry is misleading. Instead, the paper argues we should look at how the facility contributed to the ecosystem that made such discoveries possible. Another challenge is valuing things that cannot be bought or sold. How do you put a price on the diplomatic channels opened when scientists from opposing political blocs worked together during the Cold War? Or the value of inspiring a generation of students to become scientists? These impacts are real and vital, but current methods struggle to measure them in dollars.

The way these results are communicated is just as important as the numbers themselves. There is a persistent pressure to boil down a complex evaluation into a single, catchy number, such as a multiplier that claims every dollar spent returns ten dollars to the economy. The authors warn that this is dangerous. It creates a false sense of precision and encourages researchers to tweak their models until they get a favorable number. It also hides the uncertainty that is inherent in predicting the future. A more honest approach is to present a range of possible outcomes, acknowledging that the future is not a single point but a probability. This transparency allows policymakers to understand the risks and rewards without being misled by a false sense of certainty.

Perhaps the most significant recommendation in the paper is that evaluation must be treated as a life-cycle activity, not a one-off event. Currently, many facilities only think about evaluation after they are built, which creates a data gap. By the time they want to measure their impact, the baseline data is gone, and the connections between the facility and the world have become hard to trace. The authors suggest that data collection for evaluation should be built into the facility's design from the start, just like the machinery itself. This means keeping detailed records of who works there, what they buy, and what they publish, organized in a way that economists can actually use. Some facilities, like the experiments at the Large Hadron Collider, already curate their scientific data with this level of care; the paper argues they should do the same for their socio-economic data. This would allow for a continuous story of impact, updating as the facility matures, rather than a fragmented set of reports.

The paper also touches on the political nature of evaluation. Deciding what counts as a success and what does not is not just a mathematical calculation; it involves value judgments. There is a risk that facility managers might be tempted to shape their evaluations to make their projects look better, focusing on short-term, easy-to-measure wins while ignoring long-term, transformative discoveries that take decades to mature. To counter this, the authors point to a model used in France, where an independent office reviews the quality of economic evaluations for major government projects. This independent scrutiny helps ensure that the analysis is honest and not just a tool for political justification.

Ultimately, the authors conclude that we must embrace a sense of humility in how we measure the value of big science. The tools we have are improving, but they are not perfect. They struggle with the long time horizons of scientific discovery and the intangible benefits that define a healthy society. Acknowledging these limits is not a sign of weakness; it is a professional necessity. By treating evaluation as an ongoing, transparent, and integrated part of the scientific process, we can better understand the true return on our investment in big science. This shift would move us away from a culture of compliance, where reports are filed to satisfy a requirement, toward a culture of learning, where the insights gained help shape better science and better policy for the future. The goal is not to prove that every dollar spent is a guaranteed profit, but to build a system where we can honestly see how these massive engines of discovery are driving progress for all of us.