Sustainable Multi-Agent Crowdsourcing via Physics-Informed Bandits

1. Problem Definition: The Four-Way Dilemma

The paper addresses a critical challenge in modern digital gig economies and spatial crowdsourcing: sustainable agentic allocation. The authors formalize a "Four-Way Dilemma" involving four competing objectives that existing methods fail to balance simultaneously:

1. Allocation Quality: Maximizing task success rates.
2. Workforce Sustainability: Preventing worker burnout (fatigue accumulation).
3. Operational Feasibility: Avoiding the need for 100% workforce utilization (which is costly and logistically difficult).
4. Strategic Contractor Agency: Accounting for the fact that workers are rational agents who strategically declare their availability to protect their long-term earnings, rather than passive "arms" in a bandit problem.

Limitations of Existing Approaches:

• Greedy Heuristics & MCDM (e.g., TOPSIS): Achieve high Day-1 quality but are "fatigue-blind." They over-exploit top performers, leading to catastrophic burnout and severe workload imbalance.
• Standard Bandits (e.g., LinUCB, Thompson Sampling): Effectively balance workload to prevent burnout but only by engaging 100% of the workforce. This is operationally infeasible in real markets. They also suffer from high cold-start regret (requiring hundreds of interactions to converge).
• Passive Assumption: All prior work treats contractors as passive entities with fixed parameters, ignoring their strategic decision-making regarding load acceptance.

2. Methodology

The authors propose a solution consisting of two main components: a new simulation environment and a novel allocation algorithm.

A. The FORGE Simulator (K+1 Multi-Agent System)

To rigorously test allocation policies, the authors introduce FORGE, a physics-grounded simulator that extends previous frameworks (like COALESCE) with three critical features:

• Restless Fatigue Dynamics: Workers have continuous fatigue states ($f_{t,k} \in [0,1]$) that evolve autonomously. If fatigue exceeds a threshold ($\theta_{burnout}$), task success probability collapses.
• Strategic Contractor Agency: Before the allocator acts, each contractor $k$ declares a load acceptance multiplier ($a^c_{t,k} \in \{0.5, 1.0\}$).
  • $1.0$: Full availability (accepts full load).
  • $0.5$: Partial availability (accepts task but requests reduced load, accumulating half the fatigue).
  • Contractors use a threshold policy to switch to $0.5$ when fatigue approaches the burnout limit, creating a Stackelberg game where contractors act first, and the allocator responds.
• Endogenous Surge Pricing: Prices increase with frequent selection, creating economic feedback loops.

B. The Neural-Linear UCB Allocator with Physics-Informed Prior

The proposed allocation agent is a Hybrid Contextual Bandit designed to operate within the FORGE environment.

1. Two-Tower Neural Architecture:

   • Uses a Two-Tower network to map high-dimensional task queries and contractor profiles into a shared 64-dimensional embedding space.
   • This captures non-linear interactions between task requirements and dynamic contractor states (fatigue, price, reputation) better than linear models like LinUCB.
   • The contractor's strategic availability signal ($a^c_{t,k}$) is appended as a scalar to the context vector, allowing the network to learn availability as a leading indicator of fatigue.

2. Physics-Informed Covariance Prior (Offline-to-Online Transfer):

   • Problem: Standard bandits start with random weights and an identity covariance matrix, leading to high cold-start regret.
   • Solution: The authors pre-compute a Physics Prior using offline interactions from the FORGE simulator.
     • Weight Initialization ($\theta_0$): Trained on synthetic data to learn the structural "skill-cluster geometry" of the workforce (ignoring specific IDs and transient fatigue).
     • Covariance Prior ($A_0^{-1}$): A pre-computed Gram matrix of feature embeddings that "warps" the UCB confidence ellipsoid. It tells the agent which contractors are structurally similar and reduces uncertainty for known skill clusters immediately at $t=1$.
   • Effect: The agent enters the live environment with a "geometry-aware belief state," reducing cold-start regret without needing live interactions to learn basic skill mappings.

3. Hybrid Fusion Strategy:

   • The final selection score combines the Neural-Linear UCB score (for long-term learning and fatigue awareness) with a decaying TOPSIS score (for Day-1 constraint satisfaction).
   • $U_{t,k} = \text{UCB}_{nl} + \eta_t \cdot C_{t,k}$, where $\eta_t$ decays over time, allowing the system to transition from rule-based safety to learned adaptability.

3. Key Contributions

1. Empirical Diagnosis: Demonstrated that existing methods fail to solve the four-way dilemma; heuristics cause burnout, while standard bandits require infeasible 100% utilization.
2. Strategic Multi-Agent Modeling: Introduced a framework where contractors are rational agents making strategic availability decisions, transforming the problem from a passive RMAB to a Stackelberg game.
3. Neural-Linear UCB Architecture: Developed an agent that handles high-dimensional, non-linear fatigue-capability interactions while maintaining the sample efficiency of linear bandits.
4. Physics-Informed Covariance Prior: A novel offline-to-online transfer method that injects structural knowledge (skill geometry) into the UCB exploration landscape, significantly reducing cold-start regret and improving robustness to noise and turnover.

4. Experimental Results

Experiments were conducted over $T=200$ episodes (simulating one week of operation) with $K=100$ contractors.

• Performance vs. Baselines:

  • Reward: The proposed method (Hybrid + Prior) achieved the highest late-stage reward among non-oracle methods (LRew = 0.555 ± 0.041), outperforming Greedy, TOPSIS, LinUCB, and Thompson Sampling.
  • Regret: It achieved the lowest early regret (26.40 ± 4.90), demonstrating the effectiveness of the Physics Prior in the cold-start phase.
  • Utilization: It achieved these results with only 7.6% workforce utilization, whereas bandit methods required ~97–100% utilization to avoid burnout.
  • Burnout: It maintained moderate burnout levels (25.8 events) comparable to targeted routing, significantly better than the 28.8 events of TOPSIS.

• Robustness:

  • Turnover: The method remained robust under up to 50% workforce turnover. Interestingly, at high turnover (50%), the "Prior" variant outperformed the "No Prior" variant because the influx of rested workers aligned well with the prior's geometry.
  • Noise: The method maintained stability under observation noise ($\sigma = 0.20$).
  • Surge Stress: Under high traffic surge (2.0x), the method reduced burnout events by 12–22% compared to TOPSIS.

5. Significance

This paper represents a significant shift in crowdsourcing allocation research by:

• Bridging the Gap: It successfully balances the trade-off between exploitation (high quality) and sustainability (low burnout) without the operational cost of 100% workforce engagement.
• Modeling Human Agency: It moves beyond treating workers as passive resources, acknowledging and leveraging their strategic self-protection behaviors.
• Solving Cold-Start: The Physics-Informed Prior offers a practical solution to the cold-start problem in dynamic, non-stationary environments, enabling high performance from the very first allocation.
• Pareto Optimality: The method occupies a unique point on the Burnout–Reward–Utilization surface, achieving high rewards and low burnout with selective utilization—a combination no existing baseline could achieve.

In conclusion, the proposed FORGE simulator and Neural-Linear UCB with Physics Prior provide a robust, scalable framework for sustainable crowdsourcing that is ready for real-world deployment where worker fatigue and strategic behavior are critical factors.