When Crowd Size Stops Mattering: Characterizing the Ceiling Effect in Crowd-Sourced Continuous Control — A Simulation Study

Technical Summary: When Crowd Size Stops Mattering

Problem Statement
Crowd-sourced continuous control systems enable distributed participants to collectively generate real-time commands for a shared agent (e.g., in robot swarms or interactive media). While the "wisdom-of-crowds" principle suggests that aggregating more independent judgments reduces variance and improves performance, this assumption is untested in real-time continuous control under adversarial conditions. Unlike static decision-making, continuous control requires temporal synchronization and robustness to disruptive inputs. A companion study established that operating speed is the dominant factor in performance variance; this study investigates whether increasing crowd size ($N$) provides a secondary benefit, and if so, whether that benefit is monotonic or conditional on adversarial pressure and trajectory geometry.

Methodology
The study employs a controlled simulation framework to systematically investigate the joint effects of crowd size, adversarial participation ratio ($tr$), and trajectory geometry.

• System Model: A discrete-time simulation where $N$ participants vote on directional commands. The agent's state evolves based on the vector mean of these votes, subject to a fixed response delay ($\tau \approx 433$ ms) and exponential velocity smoothing.
• Variables:
  • Crowd Size ($N$): 12 levels ranging from 5 to 200.
  • Adversarial Ratio ($tr$): 10 levels ranging from 5% to 40%.
  • Trajectory Geometries: Four types: Circle (smooth), Square (corners), Lemniscate (varying curvature), and Zigzag (rapid reversals).
  • Adversary Models: Three models were tested: Uniform Random ($T0$), Committed Anti-Target ($T1$), and Persistent Stale Anti-Target ($T2$).
• Experimental Design: The study utilized a dense grid of conditions with $MC=50$ Monte Carlo realizations per cell (72,000 total runs). Statistical analysis included two-way ANOVA on raw data, ceiling-model fitting ($RMSE = a + b/\sqrt{N}$), and change-point analysis to identify transition thresholds.
• Validation: Robustness checks included varying operating speeds, manipulating lag-determining parameters (delay and smoothing), and testing behavioral sensitivity variants (e.g., fatigue, imitation).

Key Contributions

1. Adversarial Transition-Region Characterization: The study empirically identifies a gradual transition region around $tr \approx 10–20\%$ (centered near 15%). Below this threshold, increasing crowd size yields negligible improvement for smooth trajectories. Above this threshold, increasing $N$ produces measurable Root Mean Square Error (RMSE) reduction.
2. Interaction Quantification: The interaction between crowd size and adversarial ratio is substantial. For the circle trajectory, this interaction explains $\eta^2 = 21.8\%$ of the total RMSE variance; for the square, $\eta^2 = 9.6\%$. This confirms that crowd-size benefits are conditional on adversarial pressure.
3. Structural $N$-Independence: The study identifies a geometry-imposed structural limit. For rapidly reversing zigzag trajectories, crowd size exhibits practical independence across all tested conditions ($\Delta RMSE = 0.07–1.96\%$ from $N=5$ to $N=200$). This is attributed to a "lag-domination" mechanism where response delay, rather than consensus accuracy, limits performance.

Results

• Smooth Trajectories (Circle, Square, Lemniscate): A "ceiling effect" is observed where performance scales with $N$ only after the adversarial ratio exceeds the transition threshold. For the circle trajectory, the ceiling-model coefficient $b$ increases 17-fold as $tr$ rises from 5% to 40%.
• Zigzag Trajectory: Performance remains nearly flat regardless of crowd size or adversarial ratio. Time-resolved analysis indicates that tracking error is dominated by response lag during rapid directional reversals, rendering additional consensus accuracy ineffective.
• Adversarial Models: Coordinated persistent adversaries ($T2$) significantly suppress the crowd-size benefit on smooth trajectories (reducing the $N=5$ to $N=200$ RMSE reduction from 29% to 2.5%).
• Speed and Lag Sensitivity: While operating speed determines the absolute error floor, the qualitative scaling structure (whether $N$ helps) is preserved across different speeds. Controlled manipulation of lag parameters confirms that reducing effective lag shifts the transition threshold to lower adversarial ratios and strengthens $N$-dependence.

Significance and Claims
The paper positions crowd size not as a universally beneficial design factor, but as a secondary, conditional factor in collective continuous control. The findings suggest that:

• Simply increasing participation does not guarantee improved control quality.
• System designers must consider the expected adversarial participation level and task geometry.
• For smooth tasks, increasing crowd size above $N \approx 20–30$ is beneficial only once the system enters the high-adversarial regime ($tr \ge 15\%$).
• For rapidly reversing tasks, increasing participation yields negligible gains regardless of adversarial conditions due to structural lag limits.

The study explicitly notes its limitations: it is a simulation-based characterization without human-participant validation. The behavioral model captures stochastic variations but not complex social dynamics or fully coordinated Byzantine protocols. Consequently, the findings are presented as an internal-validity-oriented step to isolate parameter relationships, with ecological validation requiring future human-participant studies. The transition threshold ($tr \approx 10–20\%$) is an empirical observation within the tested range and may shift under different system parameters.