Coherence, charity and triangulation in statistical modelling
This paper critiques traditional Bayesian distinctions by applying Donald Davidson's philosophy of radical interpretation to argue that statistical modeling should be viewed as a unified belief system governed by the constraints of coherence, charity, and triangulation, thereby reframing practices like prior elicitation and model checking as efforts to align beliefs with shared human understanding and reality.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
The Invisible Map and the Shared World
Imagine you are trying to navigate a city you've never visited. You have a map, but the map isn't the city itself; it's a set of symbols, lines, and colors that represent the streets, parks, and buildings. In the world of statistics, this map is called a "model," and the city is the "data" we collect from the real world. For a long time, scientists and mathematicians have argued about the relationship between the map and the city. Some thought the city was a fixed, objective reality waiting to be measured, and the map was just a tool we used to copy it. Others thought the map was entirely in our heads, a subjective guess that had nothing to do with the real streets outside.
This paper dives into that debate, specifically within the field of Bayesian statistics, which is a way of thinking about probability as a "degree of belief." Instead of asking, "What is the true, unchangeable chance of rain?" Bayesian thinking asks, "How strongly do I believe it will rain, given what I know?" The paper challenges the idea that there is a hard wall between our internal beliefs (the map) and the external world (the city). It suggests that we can't really separate the two because our "data" is already shaped by the "model" we use to look at it. To make sense of this, the author uses three guiding principles: Coherence (your map must make sense internally, like a puzzle where all the pieces fit), Charity (you should assume other people's maps are mostly right so you can talk to them), and Triangulation (you need at least two people looking at the same thing from different angles to figure out where the real city is).
The Paper's Big Idea: It's All One Big Belief System
In this paper, David Sumpter argues that we need to stop thinking of statistics as a game of "updating" a model with new data, like pouring water into a bucket. Instead, he suggests that our entire belief system—our model, our data, and our predictions—is one giant, interconnected web. He uses a running example of a football (soccer) match to show how this works. Imagine three people watching the same game: Joe, who writes down every single shot; Jen, who only sees the score and the number of shots on her phone; and Jane, who has never watched a game but has heard for years that the team scores about one goal for every ten shots.
The paper shows that for Joe, Jen, and Jane, the line between "data" and "model" is blurry. Joe's raw list of shots is data to him, but to Jane, her belief that "it's one in ten" is her data. Jen's running totals are a mix of both. The author argues that there is no "pure" data waiting out there in the universe to be discovered. Instead, what we call data is just the leftover bits of the models we've already built. When we look at a football shot, we've already decided what counts as a "shot" and what counts as a "goal" before we even write down the number.
The paper explicitly rules out the idea that Bayes' theorem is a magical machine where data comes from the outside and updates a model from the inside. The author suggests this is a misunderstanding. Instead, Bayes' theorem is just a rule that describes how the different parts of your own belief system hang together. It's like a rule in a video game that says, "If you have this many coins and this many lives, your score must be X." The game doesn't "update" your score from the outside; the score is just a relationship between the coins and lives you already have. The paper suggests that real learning happens not by tweaking a single number, but by realizing your whole map is wrong and building a new one.
To fix this, the paper proposes that we need to add two new rules to our statistical maps, on top of the usual rule of "coherence" (making sure the math doesn't contradict itself). The first is Charity. This means that when we build a model, we should assume that other people are mostly right and that our model should make sense to them. If you invent a way to count football goals that no one else understands, your model isn't "braver"; it's just useless. The second rule is Triangulation. This means we need to check our beliefs against other people and the shared world. Just like two people looking at a mountain from different sides can agree on where the peak is, statisticians need to compare their models with others and with the real world to make sure they aren't just making things up.
The paper uses the story of Jen, the daughter, to show how this works in real life. She noticed something her dad (Joe) missed: the team seemed to miss more often right after scoring a goal. She didn't just "update" a number; she realized her whole way of looking at the game needed a tweak. She used the shared data (her dad's notes) and her own observations to build a better belief system. The author suggests that the best statisticians—like the "superforecasters" who predict world events—are the ones who do this best. They don't just stick to their own internal logic; they constantly check their beliefs against what others think and what actually happens in the world.
The paper concludes with a playful twist on a famous saying by statistician George Box, who once said, "All models are wrong, but some are useful." The author suggests we should flip this: "Most models are correct, that's why they are useful." The idea is that if a model is useful, it must already be mostly right about the world. It's not a broken map that we are trying to fix; it's a map that is already pretty good, and our job is to make it even better by listening to others and looking at the world together. The paper doesn't claim to have solved all of statistics or proved a new mathematical law. Instead, it suggests that by thinking about our beliefs as a shared, charitable, and triangulated system, we can do better science and make better predictions. It's a call to stop treating our models as isolated islands and start treating them as bridges between us and the world.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.