Fairness in Multi-Agent Systems for Software Engineering: An SDLC-Oriented Rapid Review

1. Problem Statement

The integration of Transformer-based Large Language Models (LLMs) and Multi-Agent Systems (MAS) into the Software Development Life Cycle (SDLC) is accelerating, moving from single-agent assistants to collaborative "Software 3.0" architectures. However, the fairness implications of these systems remain critically underexplored.

While fairness in single-model LLMs is a growing field, the unique dynamics of MAS—where multiple agents interact, debate, delegate, and reach consensus—introduce new failure modes. These include bias amplification, group conformity (suppressing minority viewpoints), and collective misalignment. The paper identifies a lack of systematic understanding regarding:

• How fairness is defined and measured specifically in MAS contexts.
• What specific harms arise at different stages of the SDLC (Requirements, Design, Testing, Maintenance).
• The gap between theoretical fairness frameworks and deployable, fairness-assured software systems.

2. Methodology

The authors conducted a Rapid Review following the methodology by Cartaxo, prioritizing timely synthesis over exhaustive coverage to address the fast-paced evolution of LLM-based MAS.

• Search Strategy:
  • Databases: ACM Digital Library, IEEE Xplore, and Google Scholar.
  • Time Span: 2017–2025.
  • Keywords: Combinations of "fairness/bias," "multi-agent/LLM," and "software engineering/SDLC."
  • Snowballing: One round of backward and forward snowballing was applied to seed papers.
• Selection Process:
  • Initial Pool: 350 papers identified.
  • Screening: Multi-stage screening (Title/Abstract, Partial Text) using strict inclusion/exclusion criteria.
  • Final Corpus: 18 studies retained for analysis.
• Inclusion Criteria: Empirical or conceptual work on MAS with a focus on fairness, bias, or accountability relevant to SE workflows or inter-agent interactions.
• Analysis Approach:
  • Structural Analysis: Data extraction focused on definitions, metrics, harms, and SDLC relevance.
  • Mapping: For studies not explicitly in SE, a functional mapping scheme was used to align agent interactions (e.g., debate, consensus) with SDLC stages (e.g., Testing, Design).
  • Synthesis: Thematic synthesis to identify recurring patterns, gaps, and contradictions.

3. Key Contributions

The paper makes three primary technical contributions:

1. Taxonomy of Fairness in MAS: It categorizes how fairness is currently conceptualized in the literature into three distinct, often non-overlapping strands:
   • Bias Reduction: Focus on demographic parity and stereotype reduction.
   • Ethical/Trustworthy AI: Focus on normative principles (transparency, accountability) aligned with regulations like the EU AI Act.
   • Inter-Agent Dynamics: Focus on interactional fairness (conformity, amplification, role distribution).
2. SDLC-Harm Mapping: It provides a structured mapping of reported harms to specific SDLC stages, revealing where evidence is strong (Requirements, Testing) and where it is weak (Code Review, Debugging).
3. Identification of Critical Gaps: It highlights three persistent barriers preventing the deployment of fair MAS:
   • Fragmented evaluation practices limiting comparability.
   • Lack of generalization due to narrow experimental configurations.
   • Scarce, weakly evaluated mitigation mechanisms for real-world workflows.

4. Key Results

A. Fairness Definitions and Measurement (RQ1)

The review found that fairness is measured in four distinct categories (C1–C4), but these are rarely integrated:

• C1 (Benchmark Performance): Uses standard accuracy/F1 scores on bias benchmarks (e.g., BBQ, MALIBU) as a proxy for fairness.
• C2 (Group Disparity): Statistical comparisons of outcomes across protected attributes (gender, race, etc.).
• C3 (Interactional/MAS Behavior): Metrics specific to MAS, such as conformity rates, bias amplification over interaction rounds, and collective misalignment.
• C4 (Conceptual/Normative): Discusses fairness as a governance principle (e.g., explainability, sustainability) without explicit metrics.
• Finding: There is a disconnect. Bias reduction studies (C1/C2) ignore interactional effects (C3), while governance studies (C4) lack operational metrics.

B. Biases and Harms Across the SDLC (RQ2)

The authors identified four main categories of harm and mapped them to the SDLC:

• Representation Harms: Stereotypes and marginalization of protected groups.
• Quality of Service (QoS) Harms: Unequal accuracy, hallucinations, or unsafe recommendations.
• Security/Privacy Harms: Prompt injection, data exfiltration, and cascading attacks.
• Governance/Trust Harms: Opaque decision paths and unclear responsibility.
• SDLC Distribution:
  • Requirements: Bias in scenario definition and constraint encoding.
  • Design: Architectural choices (e.g., role assignment) that propagate stereotypes.
  • Testing: Benchmarking for conversational bias (currently isolated from mainstream SE testing).
  • Maintenance: Runtime failures, drift, and operational security issues.
• Critical Observation: Evidence is heavily concentrated in Requirements and Testing, while Code Review and Debugging receive limited attention.

C. Research Gaps (RQ3)

Three major gaps prevent the field from supporting deployable systems:

1. Evaluation Infrastructure: Studies often conflate agent interaction with task choice. Variables like coordination protocols (debate vs. voting) and agent counts vary wildly, making cross-study comparison impossible.
2. Generalization: Results are often tied to specific configurations (one model, one language). There is a lack of understanding regarding intersectional biases and how specialization/division of labor in MAS triggers new inequities.
3. Mitigation & Integration: The literature is rich in problem identification but poor in deployable mitigations. Few studies implement ablation studies to separate "agent effects" from "prompting effects" or evaluate the computational cost and scalability of fairness interventions.

5. Significance and Implications

This paper serves as a critical wake-up call for the Software Engineering and AI communities.

• For Researchers: It calls for the development of MAS-aware benchmarks that specifically test interactional dynamics (e.g., bias amplification over rounds) rather than just static model outputs. It advocates for consistent evaluation protocols to enable comparability.
• For Practitioners: It highlights that current MAS architectures are not yet ready for fairness-assured deployment in critical SDLC stages. Developers must be aware that "collaborative" AI agents may inadvertently amplify biases or suppress minority viewpoints through consensus mechanisms.
• For Governance: It emphasizes the need for lifecycle-spanning governance, moving beyond high-level ethical principles to concrete, measurable controls embedded in the software development pipeline (e.g., automated fairness checks in CI/CD).

Conclusion: The current state of MAS fairness research is fragmented. To achieve "Software 3.0" that is both intelligent and equitable, the field must move from isolated, ad-hoc evaluations to standardized, lifecycle-integrated frameworks that address the unique emergent properties of multi-agent interactions.