Evaluation Bias and Epistemic Inequality in Global Software Development
This mixed-methods study of 48 software engineers across East Africa and Northwestern Europe reveals that despite comparable technical performance, systemic evaluation biases lead European participants to systematically underestimate the competence of their East African peers, highlighting significant epistemic inequalities in global software development.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine the world of software engineering as a massive, global cooking competition. Usually, judges in fancy kitchens in Northwestern Europe (like Sweden and the Netherlands) taste dishes from chefs in East Africa (Rwanda and Uganda) and assume the African dishes will be "less seasoned" or "lower quality" because they come from a different region. They expect the European chefs to be the masters of the craft.
But this paper, written by researchers Sam Khosravi and Amir H. Payberah, decided to flip the script. Instead of just tasting the final meal, they put 48 chefs from both regions in the same kitchen, gave them the exact same ingredients and recipes, and watched how they cooked, how they talked to each other, and how they judged one another.
Here is what they found, served up with a side of reality.
The Main Dish: Who Actually Cooked Better?
The researchers set up three specific challenges to test the chefs' skills:
- The "Two-Sum" Puzzle: A coding task where you have to find pairs of numbers that add up to a target. It's like finding two specific spices that make a perfect flavor combo.
- The "URL Shortener" Design: A system design challenge where chefs had to sketch out how to build a website that turns long links into short ones (like Bit.ly). This tests their ability to plan a big kitchen layout.
- The "Security Check": A code review where chefs had to find hidden traps (bugs) in a login system that was deliberately broken.
The Surprise: When the European judges looked at the results, they expected the European chefs to win easily. But the data showed something different.
- The Senior Chefs: The senior chefs from Rwanda often cooked at a level that was comparable to their European peers. In fact, when it came to spotting security bugs (the "Security Check"), the senior Rwandan chefs actually found more issues than the Dutch chefs and nearly matched the Swedish chefs.
- The Junior Chefs: The junior chefs (those with less than 3 years of experience) showed a wider gap. The Rwandan juniors struggled more, often failing to finish the tasks, while the European juniors were more consistent.
- The "Uganda" Outlier: The researchers had to set aside the data from Uganda for the coding tasks because some participants used AI tools or internet resources, which broke the rules of the "no cheating" test.
The Verdict: The paper suggests that the idea that African engineers are inherently less skilled is false. Instead, the difference comes down to experience and training. The top-tier Rwandan seniors are just as good as the Europeans, but the "average" looks lower because the training systems in Rwanda are uneven. Some chefs are world-class; others are still learning the basics.
The Flavor of Communication: Direct vs. Supportive
Next, the researchers looked at how the chefs talked to each other while reviewing the code. This is like listening to how critics talk about a dish.
- The European Style: The chefs from Sweden and the Netherlands were like tough food critics. They were very direct. If a dish was bad, they said, "This is wrong, fix it." Their comments were often neutral or slightly negative, which they saw as "constructive."
- The East African Style: The chefs from Rwanda and Uganda were like supportive friends. They used a lot of positive words. Their comments were often encouraging, like "Good start, but maybe try this."
- The Numbers: When the researchers ran a "sentiment analysis" (a computer program that measures tone), the Rwandan and Ugandan comments had a positive compound score of +0.541 and +0.919 respectively. The Dutch comments were negative (-0.361), and the Swedish ones were almost neutral (+0.007).
The paper argues that neither style is "better." The European style prioritizes efficiency and fixing errors fast. The East African style prioritizes keeping the team happy and working together. It's just a different way of cooking.
The Blind Taste Test: Who Judges Whom?
Here is where the story gets a bit sad. The researchers asked the chefs to rate each other's skills before they saw the test results.
- The European Judges: The Swedish and Dutch chefs rated themselves as the best. They rated the Rwandan and Ugandan chefs as much worse, often giving them low scores. They assumed the African chefs had "weaker education" or "bad infrastructure."
- The Reality Check: Even though the Rwandan seniors had proven they could cook just as well (or better at spotting bugs), the European judges still thought they were inferior.
- The African Judges: The Rwandan chefs rated themselves as equal to the Europeans but slightly better than the Ugandans. The Ugandan chefs rated the Europeans as the best, showing they had internalized the idea that Europeans were superior.
The paper explicitly rules out the idea that this bias is based on actual skill differences. The bias is a perception gap. The European judges are "underestimating" the African chefs, even when the evidence (the test scores) says they are wrong.
The Kitchen Workflow: Planning vs. Fixing
Finally, the researchers asked the chefs how they spend their time.
- European Chefs: They spend a huge chunk of their time (about 30% of their total effort) on the very beginning: planning, designing, and writing the code. They want to get it perfect before they start cooking.
- Rwandan Chefs: They spend less time on the initial design and more time on the end: deployment, checking the work, and fixing bugs after the dish is served. They spend about 12.9% of their time on "post-deploy verification" and 11.0% on "build & test automation."
The paper suggests this isn't because one group is lazy. It's a different strategy. Europeans try to prevent mistakes by planning hard. Rwandans try to catch mistakes by checking hard. Both work, but they look different.
The Bottom Line
This paper doesn't say that "Africa is now the best at coding" or that "Europe is failing." It says that the stereotype that African engineers are less capable is a myth.
The data shows that:
- Senior Rwandan engineers are just as capable as their European peers in many areas, especially security.
- European engineers systematically underestimate their African counterparts, creating a unfair barrier to trust and opportunity.
- Different cultures have different strengths: Europe is great at upfront design; East Africa is great at collaborative verification and supportive teamwork.
The authors suggest that if we stop judging based on where someone is from and start looking at what they can actually do, the global software kitchen could become a much more powerful and fair place. But until we fix those "blind taste tests" in our heads, we might keep missing out on some of the best chefs in the world.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.