CulTrace: Tracing Internal Cultural Reasoning in Large Language Models
The paper introduces CulTrace, a mechanistic interpretability method that reveals LLMs process cultural reasoning through a consistent three-stage trajectory of domain engagement, culture resolution, and answer selection, while also exposing imbalances where models struggle with less-represented cultures.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Large language models are the engines behind many of the artificial intelligence tools we use today, capable of generating human-like text on almost any topic. These systems are trained on vast amounts of data from the internet, learning to predict the next word in a sentence based on patterns they have seen before. Because this training data comes from all over the world, these models are increasingly expected to understand and respect the nuances of different cultures, from local customs to regional foods. However, when these models get cultural details wrong, it is often difficult to understand why. Traditional testing methods only look at the final answer a model gives, like grading a student's test without seeing their work. This approach tells us if the answer is correct, but it hides the thought process that led to it, leaving researchers in the dark about where the confusion actually happens inside the machine.
To solve this, a team of researchers from the University of Copenhagen and KAIST developed a new way to look inside these models. They created a method called CulTrace, which acts like a window into the model's internal thinking. Instead of just waiting for the final output, this technique allows researchers to read the model's "thoughts" as they happen, layer by layer. Imagine a large language model as a deep factory with many stages of production. In the early stages, the raw materials are sorted and prepared; in the middle stages, they are shaped and refined; and in the final stages, the product is assembled and packaged. CulTrace lets the researchers peek into every single stage of this factory to see exactly what the model is focusing on at each step. They found that the model does not simply jump to an answer. Instead, it follows a specific path: first, it figures out the general topic of the question, then it tries to identify the specific culture involved, and finally, it narrows down to the correct answer.
The researchers tested this method on three different popular language models using questions about ten distinct cultures, ranging from South Korea and Spain to Ethiopia and Algeria. They asked the models questions about everyday cultural knowledge, such as what snacks are popular in shopping malls in South Korea or what traditional drinks are associated with specific regions. By decoding the model's internal signals at every layer, they could watch the reasoning unfold in real time. What they discovered was a consistent pattern of how these models handle cultural information. The models always start by understanding the broad subject, like "food" or "holidays." Then, they move to identifying the culture. However, this is where the process often gets messy. Before settling on the correct culture, the models frequently wander into the territory of a nearby or similar culture. For instance, when asked about Algeria, the model might first think about North Africa or France. When asked about Iran, it might initially consider the broader Middle East.
This tendency to drift toward a "default" or a neighboring culture is where errors often begin. The study showed that for well-represented cultures like the United States, China, or South Korea, the model identifies the correct culture relatively quickly, often within the middle layers of its processing. But for cultures that appear less frequently in the training data, such as Algeria or Azerbaijan, the model takes much longer to find the right cultural context. It spends more time stuck in the early, confused stages, often defaulting to a generic regional label or a culture the model seems to favor by default, such as Japan for one of the models tested. This suggests that the bias is not just a final mistake in the output, but a fundamental issue in how the model retrieves and refines cultural knowledge during its internal processing. The model is not simply guessing; it is actively reasoning, but its reasoning path is longer and more prone to confusion for less-represented cultures.
The implications of this discovery are significant for how we build better artificial intelligence. If the problem is that the model gets stuck in the wrong cultural neighborhood during the middle stages of its thinking, then fixing the final answer is not enough. The researchers suggest that improvements need to happen earlier in the process, specifically at the layers where the model is trying to resolve the cultural context. By understanding exactly where and how these models get confused, developers can target their training efforts more precisely. Instead of just teaching the model the right answer, they can help it learn to identify the correct culture sooner and avoid the detours that lead to misattribution. This work moves the field beyond simply checking if a model is right or wrong, offering a clear map of how cultural knowledge is built, refined, and sometimes lost within the complex layers of artificial intelligence.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.