Technical Summary: Towards Value-Constrained Credit Assignment in Fully Delegated AI Cooperatives

1. Problem Statement

The paper addresses the challenge of reward allocation in fully delegated AI cooperatives. In this setting, multiple human members are represented by autonomous agents that pool data and computation to train shared AI services. The core difficulty lies in satisfying two simultaneous, coupled requirements:

1. Value Constraints: Learning must respect heterogeneous value constraints (e.g., prohibitions on military deployment, insurance risk scoring, or requirements for demographic fairness) specific to each member.
2. Economic Incentives: The economic benefits of the resulting service must be distributed according to actual contribution.

Existing approaches often fail to couple these requirements. A reward rule ignoring value constraints may compensate members for updates they would not endorse, while standard contribution accounting in Federated Learning (FL) typically operates on aggregated updates, making it difficult to trace and filter specific update directions based on individual value profiles before contribution is measured.

2. Methodology

The authors propose a framework integrating Value-Conditioned Gradient Filtering, Traversal Learning (TL), and Online Marginal Contribution Signals.

2.1 Value-Conditioned Filtering

At each iteration $t$, an agent $i$ holding local data $D_i$ and a value profile $v_i$ computes a local gradient $\nabla L_i(\theta_t)$. Before this update is propagated, it passes through a value filter $F_i$:
$$\tilde{\nabla} L_i(\theta_t) = F_i(\nabla L_i(\theta_t), v_i)$$
The filtered gradient $\tilde{\nabla} L_i$ represents the admissible part of the update. This filter can take various forms, including rule-based constitutional checks, feasible-set projections, gradient modifications, or learned black-box admissibility models. If an update direction violates the principal's values, it is rejected (set to zero), ensuring that only value-consistent updates contribute to the cooperative model.

2.2 Traversal Learning (TL) Substrate

The framework utilizes Traversal Learning (TL) as the training substrate. Unlike standard Federated Learning (e.g., FedAvg) which aggregates client updates post-hoc, TL performs decentralized backpropagation by preserving explicit traversal and gradient paths across distributed nodes.

• Mechanism: The cooperative update is defined as $\theta_{t+1} = \theta_t - \eta \sum_{i=1}^n \tilde{\nabla} L_i(\theta_t)$.
• Advantage: By maintaining explicit agent-specific gradient paths rather than just aggregated client states, TL allows for a more transparent tracking of the path from local data exposure to admissible update to cooperative effect. This structural clarity is argued to be superior for attribution in value-constrained settings compared to aggregation-centric FL.

2.3 Marginal Contribution and Settlement

The framework defines contribution and reward allocation through a multi-step process:

1. Online Credit Signal: The contribution $c_i^t$ of agent $i$ at time $t$ is defined as the marginal improvement in cooperative validation loss $L_{val}$ induced solely by the agent's admissible update:
   $$c_i^t = L_{val}(\theta_t) - L_{val}(\theta_t - \eta \tilde{\nabla} L_i(\theta_t))$$
   If the filter rejects the update ($\tilde{\nabla} L_i = 0$), the contribution is zero.
2. Cumulative Settlement: Over an accounting horizon $H$, contributions are accumulated with optional weighting $\omega_t$:
   $$C_i(H) = \sum_{t \in H} \omega_t c_i^t$$
3. Revenue Allocation: Revenue shares $r_i(H)$ are allocated based on cumulative contributions, using a power parameter $\alpha \ge 1$ to accentuate differences:
   $$r_i(H) = \frac{(C_i(H))^\alpha}{\sum_{j=1}^n (C_j(H))^\alpha}$$
   The final payment $p_i(H)$ is the share of total revenue $R(H)$. Agents are rewarded only for updates that are both admissible and validation-improving.

3. Key Contributions

• Framework Integration: The paper integrates value-conditioned update filtering, TL-based attribution, online contribution signals, and cumulative cooperative settlement into a single distributed-learning mechanism.
• Pluralism at the Gradient Level: It introduces a mechanism where agents selectively admit or reject update directions (pluralism) rather than entire tasks, allowing for heterogeneous value constraints within a shared model.
• Attribution Substrate: The authors argue that TL offers a finer and more transparent attribution substrate than FedAvg-style FL because it preserves explicit traversal paths, enabling contribution measurement only after checking value admissibility.
• Operationalizing Pluralism: The framework moves pluralistic alignment from a theoretical goal into the training-time credit-allocation mechanism itself.

4. Results and Claims

The paper is primarily a theoretical and architectural proposal; it does not present empirical experimental results or benchmark comparisons. Instead, it makes the following claims regarding the framework's properties:

• Feasibility: The framework allows for the computation of local credit signals at each step, ensuring that cumulative settlement remains well-defined even if global convergence to a single optimum is imperfect due to heterogeneous value constraints.
• Privacy and Efficiency: The authors suggest a variant of the framework (referenced as "TLplus") that could place only a cut layer or shallow front block at member agents, transmitting masked activations to an orchestrator. This aims to preserve TL's attribution benefits while reducing client-side costs and limiting information leakage, drawing on SplitNN-style partitioning.
• Positioning: The authors explicitly state they do not claim to outperform Shapley-style valuation in fairness or existing FL incentive mechanisms. Their claim is limited to the argument that TL provides a more transparent substrate for connecting value admissibility, contribution accounting, and reward allocation.

5. Significance

The significance of this work lies in its attempt to solve the "coupled" problem of value constraints and economic incentives in AI cooperatives. By rejecting the notion of collapsing heterogeneous values into a single objective, the framework makes pluralism operational within learning dynamics. It proposes a shift from post-hoc aggregation (where contribution is hard to trace) to explicit traversal (where contribution is traceable and filterable), offering a potential path toward sustainable incentive structures for diverse, value-constrained AI cooperatives.