Fairness Dynamics in Digital Economy Platforms with Biased Ratings

1. Problem Statement

The paper addresses the pervasive issue of discrimination against marginalized groups on digital service platforms (e.g., Airbnb, Uber, eBay). While rating systems are designed to build trust and facilitate cooperation, they often perpetuate biases. Users may rate providers from marginalized groups lower than equally performing providers from dominant groups, or they may infer demographic characteristics from proxies (e.g., neighborhood) to discriminate.

The core problem is a trade-off between User Experience (UX) and Fairness:

• Platforms typically prioritize providers with high ratings to ensure users get high-quality service.
• However, if ratings are biased against a specific group, prioritizing high ratings systematically excludes marginalized providers, reducing their demand and income.
• Existing literature often identifies these biases but lacks predictive models that explain the mechanisms leading to unfair outcomes or provide actionable design recommendations for platform designers to mitigate bias while maintaining incentives for high-quality service.

2. Methodology

The authors propose an Evolutionary Game Theory (EGT) model to simulate the dynamics of a digital platform.

A. Model Structure

The interaction is modeled as a four-step process:

1. Recommendation: The platform presents a list of $k$ providers to a user. The list is constructed based on ratings and potentially demographic groups.
2. Selection: The user selects a provider from the list based on their ratings (Good vs. Bad).
3. Interaction: The provider chooses a strategy: High-effort ($H$) (cooperative, costly) or Low-effort ($L$) (defective, no cost).
4. Reporting: The user reports the outcome. Ratings are updated based on this report.

B. Key Parameters

• Population: A finite set of providers divided into a Dominant group ($D$) and a Marginalized group ($M$). The user population is assumed to be infinite (mean-field assumption).
• Rating Bias ($\epsilon$): The probability that a marginalized provider playing $H$ is falsely reported as playing $L$ (receiving a "Bad" rating).
• Rating Sensitivity ($\gamma$): The degree to which users trust the rating system. If $\gamma=1$, users only choose "Good" rated providers; if $\gamma < 1$, they may occasionally choose "Bad" rated providers.
• Consumer Involvement ($k$): The number of options a user sees before selecting.
• Algorithmic Intervention:
  • $k_G$: The number of "Good" rated providers guaranteed in the search results.
  • $k_M$: The number of marginalized providers guaranteed within the top $k_G$ results (an explicit anti-discrimination mechanism).

C. Dynamics

The model uses a frequency-dependent Moran process to simulate strategy evolution over time:

• Providers update their strategies ($H$ or $L$) based on utility comparisons with others in their group (Fermi pairwise imitation rule).
• A mutation rate ($\mu$) allows for random strategy switching.
• The system converges to a stationary distribution ($h^*$), representing the long-term probability of different states (combinations of $H$-players in each group).

D. Evaluation Metrics

1. User Experience (UX): The probability that a user is matched with a provider playing High-effort ($H$).
2. Demographic Parity Ratio (DPR): The ratio of the average utility of the marginalized group to the dominant group. A value of 1 indicates perfect fairness.
3. Pareto Optimality: Identifying design choices ($k_G, k_M$) where one cannot improve UX without worsening DPR, and vice versa.

3. Key Contributions

1. EGT Model for Platform Fairness: The authors formulate a novel EGT model that captures how different groups of agents respond to recommender systems and rating biases, moving beyond static analysis to dynamic evolutionary outcomes.
2. Identification of Necessary Conditions: The study numerically defines the necessary conditions for a platform to sufficiently incentivize high-effort behavior across all groups, highlighting the fragility of cooperation under bias.
3. Quantification of the Trade-off: The paper demonstrates that under the constraint of "blind" algorithms ($k_M = 0$), there is an inherent, unavoidable trade-off between maximizing user experience and ensuring group-level fairness.
4. Algorithmic Intervention Strategy: The authors prove that this trade-off vanishes when platforms explicitly prioritize marginalized groups ($k_M > 0$). They show that tuning the demographics of search results is a highly effective intervention that improves fairness with minimal impact on user experience.
5. Robustness to Uncertainty: The model shows that even when the exact level of rating bias ($\epsilon$) is unknown, implementing a policy with $k_M > 0$ yields better worst-case and average-case outcomes than ignoring protected characteristics.

4. Results

• Cooperation Baseline: Without a reputation system or with insufficient incentives, no providers cooperate. With a reputation system, a "separating equilibrium" can occur where the dominant group cooperates, but the marginalized group defects due to the difficulty of maintaining a high rating under bias.
• The Trade-off ($k_M = 0$): When platforms only prioritize high ratings ($k_M=0$), increasing the number of high-rated providers shown ($k_G$) improves UX but decreases DPR. This is because the marginalized group, suffering from bias, cannot achieve high ratings as easily, so they are filtered out of the top results, further reducing their opportunities to build reputation.
• The Solution ($k_M > 0$): Introducing a constraint to guarantee marginalized providers in the top results ($k_M > 0$) breaks the trade-off.
  • Fairness: Increasing $k_M$ significantly improves the Demographic Parity Ratio (DPR).
  • User Experience: UX remains largely stable because the number of high-effort providers shown does not decrease; only the demographic composition changes.
  • Thresholds: The optimal $k_M$ increases as the level of bias ($\epsilon$) increases.
• Uncertainty: Even with significant uncertainty regarding the true level of bias, a strategy of setting $k_M > 0$ (optimizing for the worst-case scenario) outperforms the status quo ($k_M = 0$) in both average and worst-case scenarios.

5. Significance

• Proactive Design: The paper argues that platform designers should not rely solely on "neutral" algorithms. Proactive anti-discrimination design (tuning search result demographics) is necessary to counteract inherent user biases.
• Policy Implications: The findings suggest that regulations or internal policies requiring platforms to ensure representation in search results (e.g., "fairness by design") can improve labor market outcomes for vulnerable groups without degrading the quality of service for consumers.
• Theoretical Contribution: It bridges the gap between evolutionary game theory and digital economics, providing a mechanistic understanding of how algorithmic choices interact with human bias to create systemic inequality.
• Practical Application: The results offer a concrete, actionable guideline for platform engineers: Do not just optimize for the highest-rated providers; explicitly weight the demographic composition of the top results to ensure marginalized groups remain in the pool of visible options.

In conclusion, the paper demonstrates that while rating systems are essential for digital economies, they are insufficient to prevent discrimination on their own. However, by strategically adjusting the recommender system to prioritize marginalized groups, platforms can achieve a "win-win" scenario where fairness is restored without compromising the user experience.