Colour Blinded by the Noise
This paper introduces a novel evaluation framework that adapts the Ishihara colourblind test to assess uncertainty visualisation methods by treating uncertainty as noise, thereby establishing foundational theory for signal suppression and resolving conflicting results in the field.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Maps are powerful tools for seeing patterns in the world, from the spread of a disease to the voting habits of a state. But every map is built on data that is never perfectly certain. When researchers count people or measure rainfall, there is always a margin of error, a range of possibilities rather than a single, hard number. The challenge for mapmakers is how to show this uncertainty without hiding the real patterns. If a map shows too much detail about the error, the true signal gets lost in the noise. If it shows no error at all, the map might trick the viewer into seeing a pattern that isn't really there, leading to bad decisions. For decades, scientists have debated the best way to balance these needs, often testing methods by asking people to interpret complex charts. This approach, however, often confuses the viewer's ability to read a chart with their ability to see the truth.
A team of researchers set out to solve this by changing the question entirely. Instead of asking people to interpret statistics, they asked a simpler question: can you see the shape? They treated uncertainty not as information to be read, but as noise that should hide false patterns. To do this, they borrowed a trick from a classic test for color blindness. In that test, a person looks at a plate of colored dots and tries to find a hidden number. If the person has normal vision, the number pops out; if they have a specific type of color blindness, the dots blend together, and the number disappears. The researchers realized they could use this same principle for maps. They created maps where the "number" was a real pattern in the data, and the "color blindness" was the statistical uncertainty. If a map was good at showing uncertainty, it would make the hidden number disappear when the data was shaky, just as color blindness makes a number disappear when the colors are too similar.
The team tested five different ways of drawing these maps. The first was the standard map, where each region is a single solid color representing the average value. The second was a two-variable map that used both color and brightness to show the average and the error. The third was a special palette that blended colors together as the error grew larger. The fourth and fifth methods were more radical: instead of showing a single average, they showed the data as a collection of many tiny dots or transparent layers, essentially painting the map with a sample of possible outcomes. They generated hundreds of these maps, each with a hidden number made of regions that were slightly different from the background. They varied the strength of the pattern and the amount of uncertainty in the data to see which maps made the number visible only when it was statistically real.
The results were clear and surprising. The standard map and the two-variable map failed to hide the false patterns. No matter how much uncertainty was in the data, these maps still showed the number clearly. They acted as if the data was perfect, even when it was not. This means that simply adding a second color or a brightness scale does not stop a viewer from seeing a pattern that might be a statistical fluke. The third method, which blended colors, did hide some patterns, but it did so in a clumsy way. It hid the number only when the uncertainty was at its absolute maximum, failing to adjust smoothly as the data became less certain.
The maps that worked best were the ones that showed the full range of possibilities. The maps made of tiny dots and the transparent layers both succeeded in hiding the number when the data was too noisy to support it, and revealing it when the pattern was strong. These methods did not rely on complex color mixing or special palettes. Instead, they relied on the simple act of showing the data as a collection of many possibilities. When the data was uncertain, the dots or layers mixed together so thoroughly that the shape of the number dissolved into the background. When the data was certain, the dots aligned to form a clear shape. This suggests that to truly show uncertainty, a map must stop trying to summarize the data into a single number and start showing the data as it really is: a collection of possibilities.
The study involved 137 people who looked at these maps on a screen and tried to identify the hidden numbers. The researchers found that the ability to see the number changed depending on the map type and the amount of uncertainty, just as the theory predicted. The maps that showed the full distribution of data were the only ones that behaved like a proper statistical test, hiding the signal when it was weak and showing it when it was strong. This finding offers a new path forward for anyone who needs to make decisions based on maps. It suggests that the best way to prevent false conclusions is not to add more colors or labels, but to let the viewer see the raw, messy reality of the data. By doing so, the map itself becomes a tool for judgment, blurring out the noise so that only the true signal remains visible.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.