Fairness-in-the-Workflow: How Machine Learning Practitioners at Big Tech Companies Approach Fairness in Recommender Systems

Through interviews with 11 practitioners at large technology companies, this paper maps the workflow of integrating fairness into recommender systems, identifying key technical and organizational challenges such as defining fairness, balancing stakeholder interests, and facilitating cross-team collaboration, while offering actionable recommendations for the community.

The Gist

Imagine you are walking through a massive, bustling digital marketplace. This isn't just any market; it's run by giant, invisible shopkeepers (the Recommender Systems) who decide what you see, what you buy, and what ideas you encounter. They are the reason you see a specific pair of shoes on Amazon or a specific political video on social media.

The problem? These invisible shopkeepers can be biased. They might accidentally favor one group of people over another, or push certain ideas so hard that they distort reality.

This paper is like a behind-the-scenes documentary featuring 11 of the actual engineers and scientists ("the mechanics") who build and maintain these digital marketplaces at giant tech companies. The researchers asked them: "How do you try to make sure your machines are fair, and what makes it so hard?"

Here is the story of their findings, explained simply:

1. The Job: Building a Fair Machine

Think of building a recommender system like cooking a giant, complex stew.

• The Ingredients (Data): You need data about users and products.
• The Recipe (The Model): You write code to decide how to mix them.
• The Taste Test (Fairness): You have to make sure the stew tastes good for everyone, not just the people who usually eat it.

The researchers found that these mechanics have a specific workflow. It starts in a quiet kitchen (Offline Development) where they gather ingredients and write recipes. Then, they serve the stew to the public (Online Development) and watch how people react.

2. The Three Big Hurdles (Technical Challenges)

Hurdle A: Defining "Fair" is Like Defining "Delicious"
In school, you learn that "fair" means everyone gets the same size slice of cake. But in the real world, it's messier.

• The Problem: If you have a dating app, is it fair to show everyone the same number of profiles? Or is it fair to show people profiles they are actually likely to date?
• The Metaphor: Imagine you are a chef. One customer says, "I want a spicy dish." Another says, "I want a sweet dish." If you make the dish "fair" by making it mild, you might upset both of them. The mechanics struggle because there is no single rulebook for what "fair" means in every situation. They often have to invent their own rules on the fly.

Hurdle B: The "Too Many Cooks" Problem
Recommender systems serve two main groups: the Users (people scrolling) and the Providers (sellers, creators, advertisers).

• The Problem: What's fair for the user might be unfair for the creator, and vice versa.
• The Metaphor: Imagine a traffic light. If you make the light stay green for the pedestrians (users) for a long time, the cars (creators) get stuck and can't deliver their goods. The mechanics have to balance these competing interests constantly. Sometimes, they have to choose who gets the "green light," and that's a really hard decision to make.

Hurdle C: The "Moving Target"
Unlike a static bridge, a recommender system changes based on how people use it.

• The Problem: If the system shows you a video you like, you click it. The system thinks, "Oh, they love this!" and shows you more of it. Soon, you are in a bubble, only seeing one type of content. This is called a Feedback Loop.
• The Metaphor: It's like a snowball rolling down a hill. It starts small, but as it rolls, it picks up more snow (data) and gets huge. By the time the mechanics realize the snowball is too big and rolling the wrong way, it's very hard to stop it. They often have to wait until the system is live to see these problems, which is like trying to fix a car while driving it at 100 mph.

3. The Office Politics (Organizational Challenges)

Challenge A: The "No Time" Crisis
The mechanics are under immense pressure to keep the system running fast and profitable.

• The Metaphor: Imagine you are a mechanic in a busy garage. The boss says, "Fix the engine so the car goes faster!" You also want to say, "Hey, let's also make sure the brakes are safe for everyone." But the boss says, "We don't have time for safety checks today; we need to get cars out the door."
• The Reality: The mechanics reported spending less than 10% of their time on fairness. It's not that they don't care; it's that "fairness" isn't seen as an emergency, so it gets pushed to the bottom of the to-do list.

Challenge B: The "Language Barrier"
The mechanics talk to two other teams: Lawyers and Fairness Experts.

• The Lawyers: They speak a clear language: "You cannot use this data, or you will get sued." The mechanics love this because the rules are black and white.
• The Fairness Experts: They speak a different language. They talk about "societal impact," "psychological bias," and "ethical nuance."
• The Metaphor: It's like a Translator trying to explain a poem to a computer. The Fairness Experts say, "This feels unfair." The Mechanics ask, "Okay, but what line of code do I change?" The two teams often struggle to understand each other because they don't share a common dictionary.

4. What Should We Do? (The Recommendations)

The paper suggests a few ways to fix this:

1. Write It Down (Documentation): Mechanics often forget what they did last year. They need better "recipe books" so they don't have to reinvent the wheel every time they try to fix a bias.
2. Start Early: Don't wait until the car is built to check the brakes. Bring the Fairness Experts into the kitchen while the recipe is being written, not just when the food is served.
3. Learn Each Other's Languages: The mechanics and the fairness experts need to learn to speak the same language. Maybe the Fairness Experts need to learn a bit of code, and the mechanics need to learn a bit of ethics.
4. Give Them Time: Companies need to officially pay mechanics to spend time thinking about fairness, just like they pay them to make the system faster.

The Bottom Line

The people building these systems want to do the right thing. They know their creations shape society. But they are stuck in a difficult spot: they are trying to build a perfectly fair machine in a world that is messy, fast-paced, and full of conflicting rules. They need better tools, more time, and a better way to talk to each other to make the digital world fair for everyone.

Technical Summary

1. Problem Statement

Recommender Systems (RS) are critical infrastructure in high-stakes domains (e-commerce, social media) but are susceptible to biases that cause large-scale societal, psychological, and ideological impacts. While academic research has proposed numerous methods to measure and mitigate bias, translating these theories into industry practice is inherently difficult.

The core problem addressed is the gap between academic fairness definitions and the operational reality of RS practitioners in "Big Tech" companies. Specifically:

• Dynamic Complexity: Unlike static classification tasks, RS operate in multi-sided markets with dynamic feedback loops, making fairness a shifting concept rather than a static metric.
• Multi-Stakeholder Conflict: RS must balance conflicting interests between providers (content creators/sellers) and end-users (consumers), where optimizing for one group often harms the other.
• Operational Barriers: Practitioners face challenges in defining fairness without legal mandates, adapting academic tools to specific contexts, and navigating organizational constraints (time, cross-team communication).

2. Methodology

The authors conducted a qualitative, semi-structured interview study to map the real-world workflows of RS practitioners.

• Participants: 11 Machine Learning practitioners (Machine Learning Engineers, Researchers, Scientists) from seven different Big Tech companies.
  • All companies serve over 1 billion customers annually.
  • Participants had an average of 5.36 years of experience.
  • Roles involved active building and maintenance of RS.
• Procedure:
  • Interviews lasted 45–60 minutes over a three-week period in 2022.
  • Research Questions (RQs):
    1. What do the technical and fairness workflows of an RS practitioner look like, and how do they intersect?
    2. What challenges do practitioners face when incorporating fairness into their technical workflows?
• Analysis:
  • Workflow Mapping: Collaborative identification of common stages in the RS development lifecycle (Offline vs. Online).
  • Thematic Analysis: Inductive coding of transcripts to identify technical and organizational challenges.
  • Positionality: The research team comprised experts in ML, HCI, and algorithmic fairness, combining industry-informed practical constraints with academic rigor.

3. Key Contributions

The paper makes three primary contributions to the HCI and ML fairness communities:

1. A Workflow Map: A detailed visualization of the RS practitioner workflow, highlighting interactions with Legal, Data, and Fairness teams across Offline (Prototyping/Internal) and Online (Collaborative) phases.
2. Challenge Taxonomy: A comprehensive identification of technical and organizational barriers to implementing fairness, distinguishing between challenges inherent to RS complexity and those arising from organizational structure.
3. Actionable Recommendations: Concrete calls to action for practitioners, organizations, and HCI researchers to better integrate fairness into RS development.

4. Key Results & Findings

A. The RS Practitioner Workflow

The authors identified a workflow consisting of two main phases with distinct fairness activities:

• Offline Development (Prototyping & Internal):
  • Data Acquisition: Legal teams approve data access; Data teams facilitate API access.
  • Prototyping: Practitioners hypothesize sources of unfairness and develop intuition-based, bespoke metrics (often reusing internal code rather than academic literature).
  • Internal Validation: Iterative testing of models and manual searching for hidden biases before deployment.
• Online Development (Collaborative):
  • Deployment & Monitoring: Models are shipped; real-world user feedback (implicit/explicit) is collected.
  • Collaboration: The Fairness Team (often called "Responsible AI") provides feedback based on internal guidelines.
  • Iterative Refinement: Practitioners manually search for "hidden" biases in deployed systems and create ad-hoc metrics to address them.

B. Technical Challenges (Within RS Teams)

1. Defining Fairness:
   • Practitioners struggle to define fairness in contexts not covered by law (e.g., dating apps vs. job boards).
   • Academic metrics are often ill-suited for RS; practitioners rely on intuition and domain-specific heuristics rather than standardized academic tools.
2. Multi-Stakeholder Trade-offs:
   • RS must balance conflicting interests (e.g., user satisfaction vs. creator exposure).
   • Practitioners face the unique challenge of prioritizing non-conflicting fairness goals (e.g., demographic parity for users AND regulatory compliance for advertisers) due to resource constraints.
3. Dynamic Environments & Feedback Loops:
   • Offline testing is insufficient because RS are shaped by user behavior.
   • Practitioners must detect "hidden" biases that only emerge after deployment via feedback loops, requiring constant, manual, ad-hoc metric creation.

C. Organizational Challenges (Cross-Team)

1. Lack of Time:
   • Practitioners spend <10% of their time on fairness. Fairness issues are rarely treated as "disruptive emergencies" compared to uptime or engagement metrics.
2. Lack of a "Fairness Lingua Franca":
   • Communication Gap: Technical teams struggle to understand feedback from Fairness teams (often composed of psychologists/sociologists) due to terminology mismatches.
   • Contrast with Legal Teams: Communication with Legal teams was described as clear and directive (binary "yes/no"), whereas Fairness feedback was often nuanced, abstract, and arrived too late in the development cycle.
3. Documentation Debt:
   • Institutional knowledge regarding fairness metrics is poorly documented, forcing practitioners to reinvent metrics for every new project.

5. Significance and Recommendations

The study highlights that fairness in RS is not just a technical problem but a socio-technical workflow problem. The disconnect between academic theory and industry practice stems from the dynamic, multi-stakeholder nature of RS and organizational silos.

Key Recommendations:

• For Practitioners & Organizations:
  • Documentation: Create extensible, modular documentation for fairness metrics to build institutional knowledge and avoid "reinventing the wheel."
  • Early Integration: Involve Fairness teams during the Prototyping phase, not just post-deployment.
  • Incentivize Research: Make reviewing academic fairness literature an explicit job responsibility to bridge the gap between theory and practice.
• For HCI Researchers:
  • Multi-Stakeholder Tools: Develop frameworks (e.g., Value-Sensitive Design) to help practitioners navigate trade-offs between conflicting stakeholder groups.
  • Translation Artifacts: Create tools that act as a "bridge" (a lingua franca) to translate abstract fairness concepts into actionable technical constraints for engineers.
  • Workflow Integration: Design tools that trigger "ethical recognition" early in the development lifecycle, moving beyond post-hoc auditing.

Conclusion:
The paper concludes that while practitioners are intrinsically motivated to improve fairness, they are hindered by a lack of time, clear guidelines, and shared terminology. Bridging the gap requires moving from ad-hoc, intuition-based fixes to structured, documented, and early-integrated workflows supported by both organizational policy and HCI-designed tools.