🔬 physics

The Fragility of Global Comparisons of Perceived Scientist Trustworthiness: Evidence from Measurement Alignment across 68 Countries/Regions

This study demonstrates that cross-national comparisons of perceived scientist trustworthiness based on observed-score averages are misleading due to a lack of measurement invariance, and that applying measurement alignment to latent variables reveals significantly different country rankings and substantive associations compared to the original findings.

Original authors: Yuke Yang, Zhihao Ma

Published 2026-01-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yuke Yang, Zhihao Ma

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you and 67 other friends are trying to rate how much you trust your local scientists. You all fill out the same 12-question survey. The original study (Cologna et al., 2025) took all these answers, averaged them up for each country, and created a giant "Trust in Scientists" leaderboard. They then used this leaderboard to see what makes people trust scientists more or less.

The Problem: The Ruler Was Broken
The authors of this new paper (Yang and Ma) looked at that study and said, "Wait a minute." They argued that the survey questions didn't work the same way in every country.

Think of it like this: If you ask a person in Country A, "Are scientists honest?" they might think of a scientist who tells the truth about data. But in Country B, that same question might make people think of a scientist who is kind and helpful. Even though the word "honest" is the same, the meaning behind the answer is different.

The original study tried to compare the average scores as if everyone was using the exact same ruler. But if one person is measuring in inches and another in centimeters, you can't just add the numbers together and say, "This country is taller than that one." The original study admitted the "ruler" wasn't perfect, but they went ahead and compared the averages anyway.

The Solution: The "Alignment" Tool
The new authors used a sophisticated statistical tool called Measurement Alignment.

Imagine you have 68 different maps of the same city, but they are all drawn at different scales and slightly rotated.

The Old Way: You just stack the maps on top of each other and guess where the landmarks are. It's messy and inaccurate.
The New Way (Alignment): You use a special algorithm to rotate, stretch, and shrink the maps just enough so that the "honesty" landmark on Map A lines up perfectly with the "honesty" landmark on Map B. You acknowledge the maps are slightly different, but you force them to align so you can make a fair comparison.

What Happened When They Re-did the Math?
When the authors applied this "alignment" tool to the data, the results changed dramatically:

The Leaderboard Flipped: In the original study, some countries were at the top and some at the bottom. After alignment, 62 out of 68 countries moved up or down the list. Some moved a lot! For example, Uruguay dropped 15 spots, while the Democratic Republic of Congo jumped up 12 spots. The "best" and "worst" countries for trusting scientists were no longer the same.
The "Why" Changed: The original study claimed that people who held "populist" views (distrusting elites) or believed in "social dominance" (wanting a strict hierarchy) trusted scientists much less.
- The New Finding: When using the aligned scores, those connections basically disappeared. The link between those political views and trusting scientists was near zero.
- What Stayed True: The only things that still strongly predicted trust were positive attitudes toward science itself (e.g., "Science is beneficial," "I trust the scientific method").

The Bottom Line
The paper argues that when you compare different countries, you can't just take a simple average of survey answers. If the questions don't mean the exact same thing everywhere, your "leaderboard" is misleading.

By using the "alignment" method to fix the broken ruler, the authors found that:

The ranking of which countries trust scientists the most is very different from what was previously reported.
The idea that political ideology (like populism) strongly drives distrust in scientists might have been an illusion caused by bad measurement.

In short: You can't compare apples and oranges just because they are both fruit. You have to make sure you are comparing the same kind of fruit, or your conclusions about which is "better" will be wrong.

Technical Summary: The Fragility of Global Comparisons of Perceived Scientist Trustworthiness

Problem Statement
This study addresses a critical methodological flaw in cross-national survey research regarding the measurement of "perceived scientist trustworthiness." It re-evaluates the findings of Cologna et al. (2025), which analyzed data from 68 countries/regions ( $N = 69,534$ ). While Cologna et al. acknowledged that their four-factor trustworthiness scale failed to satisfy metric, scalar, and strict measurement invariance (MI), they proceeded to conduct cross-national comparisons and regression analyses using weighted means of observed item scores. The authors of this paper argue that such comparisons implicitly assume cross-country comparability at the observed-score level, a condition not met when scalar invariance is absent. Without establishing measurement equivalence, comparing latent means or associations across groups risks yielding misleading or substantively ambiguous results.

Methodology
The authors re-analyzed the publicly shared dataset using a rigorous measurement alignment framework to accommodate cross-country non-invariance. The analysis followed four distinct analytical paths to test the robustness of results against weighting strategies and model specifications:

Pooled vs. Country-Specific CFA: The study compared a pooled-sample Confirmatory Factor Analysis (CFA) (analyzing all countries simultaneously) against 68 separate country-specific CFAs.
Weighting Strategies: Each CFA path was estimated both with and without survey weights to evaluate sensitivity to weighting specifications.
Measurement Alignment: Following the approach of Asparouhov and Muthén (2014) and Han (2023), the authors applied measurement alignment to the four-factor model (competence, integrity, benevolence, and openness). This method allows for a degree of non-equivalence in measurement parameters (loadings and intercepts) across groups while generating comparable latent factor scores.
Re-estimation of Substantive Models: Using the aligned latent scores (where alignment converged successfully), the authors re-ran the multilevel linear regression models to assess associations between perceived scientist trustworthiness and demographic, ideological, attitudinal, and country-level predictors.

Key Results

Measurement Invariance: Across all four analytical specifications, the data supported configural and metric invariance but consistently failed to establish scalar or strict invariance. Attempts to establish partial invariance resulted in model non-convergence.
Alignment Success: Measurement alignment yielded admissible solutions only for the competence and openness dimensions. The integrity and benevolence factors consistently failed to align (producing unstable solutions or negative $R^2$ values), likely due to the cultural and contextual complexity of these constructs in a societal setting versus the original organizational trust theory context.
Ranking Instability: When country/region rankings were recalculated using aligned latent scores (specifically for the competence and openness dimensions) rather than observed weighted means, the ordering changed for 62 of the 68 countries/regions. Notable shifts included Uruguay dropping 15 ranks, South Korea dropping 11, and the Democratic Republic of Congo rising 12 ranks.
Substantive Associations: The re-analysis fundamentally altered the conclusions regarding key predictors:
- Science-related populist attitudes: The original study reported a significant negative association with trustworthiness. In the re-analysis using aligned scores, this association was near zero and non-significant ( $\beta \approx 0.003, p = 0.775$ ).
- Social Dominance Orientation (SDO): Similarly, the significant negative association reported in the original study became non-significant in the re-analyzed models.
- Attitudes toward Science: Variables such as perceived benefit of science, willingness to be vulnerable to science, and trust in the scientific method remained strongly and positively associated with trustworthiness, consistent with the original findings.

Significance and Claims
The paper claims that cross-national comparisons based on observed-score averages are misleading when measurement equivalence is not established. The authors demonstrate that the "league-table" rankings of countries and the inferred relationships between ideological variables (like populism and SDO) and trust in scientists are highly sensitive to how measurement non-equivalence is addressed.

The study concludes that latent-variable approaches, specifically measurement alignment, provide a more defensible basis for international inference than simple observed-score aggregation. It underscores that establishing—or appropriately approximating—measurement comparability is not merely a technical detail but a prerequisite for interpreting cross-country differences in levels and effect sizes. Furthermore, the failure of the integrity factor to align suggests that the conceptualization of "trust" may not be equivalent across diverse cultural and institutional settings, calling for more extensive cross-cultural conceptual clarification and potentially qualitative validation of scale items in future research.

More like this