Sorting Methods for Online Deliberation: Towards a Principled Approach

1. Problem Statement

Online Deliberation Platforms (DPs) are increasingly used to enhance democratic participation by allowing citizens to propose, comment on, and vote for policy proposals. However, these platforms face a critical design challenge: information overload. Successful DPs often generate hundreds to thousands of proposals (e.g., 27,000+ on Madrid's Consul platform), far exceeding the capacity of any individual user to review.

The core problem addressed is the normative and technical question of how to sort these proposals for display. The authors argue that:

• Current sorting methods are often ad hoc, lacking explicit justification or alignment with democratic principles.
• The sorting method directly influences user attention, deliberation quality, and decision-making outcomes due to "position bias" (users rarely look past the first page of results).
• The dominant practice of sorting by "number of approvals" (likes) is problematic. It risks amplifying the voices of large, homogeneous groups while burying minority viewpoints, thereby undermining the inclusivity and deliberative quality essential to democratic legitimacy.

2. Methodology

The authors employ a mixed-method approach combining conceptual analysis, empirical observation, and theoretical critique:

• Conceptual Framework Development: They construct a framework to classify sorting methods based on:
  • Purpose: Distinguishing between sorting for online deliberation (browsing/debating), offline deliberation (agenda setting for meetings), and voting (ballot ordering).
  • Parameters: Categorizing inputs into Content (topic, tags, cost), Evaluation (likes, comments, conditional approvals), and Metadata (submission date, view counts, author history).
• Empirical Survey: The authors analyzed a representative sample of 8 successful DPs (including Consul, Decidim, LiquidFeedback, OpenStad, and Go Vocal). They examined:
  • The volume of proposals generated (averaging 2,724 per case).
  • The default and user-selectable sorting algorithms currently in use.
  • The distinction between sorting for deliberation vs. voting phases.
• Normative Critique: They evaluate existing methods against core principles of Deliberative Democracy (inclusivity, judgmental reflection, and dialogical exchange) and instrumental arguments (epistemic quality and perceived legitimacy).

3. Key Contributions

A. Conceptual Framework for Sorting

The paper establishes a structured taxonomy for analyzing sorting methods, moving beyond simple "popularity" metrics. It emphasizes that the choice of sorting method must be contingent on the purpose of the list (e.g., maximizing diversity for deliberation vs. reflecting majority will for voting).

B. Empirical Findings on Current Practices

• Lack of Standardization: There is no universally preferred sorting method; platforms vary wildly (random, date-based, proportional, or approval-based).
• Ad Hoc Justification: Most platforms do not explicitly justify their sorting choices, often leaving the decision to users or defaulting to "newest first" or "most approved."
• Dominance of Single Parameters: Most platforms rely on a single parameter (e.g., date or approval count) rather than integrating content, metadata, and evaluation.
• The "Approval" Trap: Sorting by approval numbers is the most common option, despite the authors' argument that it creates a "rich-get-richer" dynamic where large groups dominate the visibility of proposals.

C. Theoretical Critique of "Most Approvals First"

The authors provide a rigorous critique of sorting by approval counts, highlighting three specific failures:

1. Inclusivity Failure: Large, homogeneous groups can "bury" proposals from smaller groups, violating the principle that minorities should be heard as well as majorities.
2. Epistemic Failure: Approval counts do not necessarily correlate with quality, especially if proposals haven't been sufficiently inspected (viewed) by a diverse audience. It creates feedback loops where mediocre but popular proposals stay visible.
3. Digital Divide Amplification: Relying on raw numbers ignores the "digital divide," where certain demographics are less active online, leading to the systematic exclusion of their viewpoints.

D. Proposed Alternative Approaches

The paper advocates for holistic, integrated sorting methods that move beyond simple ranking:

• Integrated Parameters: Sorting algorithms should combine approval data with metadata (e.g., ensuring a proposal has been viewed enough times to be valid) and content diversity (ensuring different topics/regions are represented).
• Proportional Sorting: Drawing from Computational Social Choice, the authors suggest algorithms that reflect the size of supporting groups proportionally (e.g., if Group A supports 40% of proposals, 40% of the top list should be from Group A).
• Maximal Coverage: An approach that prioritizes ensuring that every distinct group of users sees at least one proposal they support near the top of the list, rather than maximizing the total number of approvals for the top few items.

4. Results and Analysis

• Current State: The survey reveals that while platforms offer variety, the default mechanisms often fail to support the specific goals of the deliberative phase. For instance, sorting by "most approved" is common even when the goal is to expose users to diverse viewpoints.
• Simulation/Logic: Using a theoretical scenario (100 users, 30 proposals, 3 distinct groups), the authors demonstrate that "Most Approvals" sorting results in the top 10 positions being occupied solely by one group's proposals, effectively silencing the other two groups. In contrast, Proportional and Maximal Coverage methods ensure all groups are represented in the top tier.
• Legitimacy: The authors argue that while "most approved" feels familiar (like social media), proportional and coverage-based methods are likely to be perceived as more legitimate in a democratic context because they align with the value of inclusivity.

5. Significance and Future Directions

This paper is significant because it shifts the discourse on digital democracy from purely technical implementation to normative design. It argues that algorithmic choices in DPs are political decisions that determine whose voice is heard.

Key Implications:

• Design Guidelines: Platform developers should move away from single-parameter sorting (especially raw approval counts) toward integrated, holistic algorithms that prioritize diversity and proportional representation during the deliberation phase.
• Research Agenda: The paper concludes by identifying five critical open questions for future research:
  1. The causal effect of sorting methods on proposal quality.
  2. The impact on perceived legitimacy.
  3. User preferences vs. actual usage of sorting options.
  4. How to transparently communicate sorting logic to users.
  5. Methods to measure and integrate "inspection depth" (how many people actually read a proposal) into sorting algorithms.

In summary, the paper provides a foundational argument that sorting is not a neutral technical feature but a core democratic mechanism. To achieve the goals of online deliberation, sorting methods must be principled, integrated, and designed to ensure the visibility of diverse viewpoints rather than merely amplifying the loudest voices.