Technical Summary: MAC – A Conversion Rate Prediction Benchmark Featuring Labels Under Multiple Attribution Mechanisms

1. Problem Definition

Conversion Rate (CVR) prediction is fundamental to online advertising, directly impacting bidding efficiency and platform revenue. Traditionally, CVR models are trained under a single-attribution assumption, where conversion labels are generated by a single mechanism (e.g., Last-Click). This paradigm limits the model's ability to capture the holistic user conversion mindset, as different attribution mechanisms (e.g., First-Click, Linear, Data-Driven Attribution) encode distinct perspectives on user intent and touchpoint contributions.

While Multi-Attribution Learning (MAL) has emerged as a promising paradigm to jointly learn from labels generated by multiple attribution mechanisms, its advancement has been stifled by a critical data bottleneck: existing public CVR datasets (e.g., Criteo, Ali-CCP, Taobao) only provide labels from a single attribution mechanism. Consequently, there is a lack of reproducible benchmarks and systematic evaluations for MAL approaches.

2. Methodology and Resources

2.1 The MAC Benchmark

To address the data gap, the authors introduce MAC (Multi-Attribution BenChmark), the first public CVR dataset featuring conversion labels from four distinct attribution mechanisms:

• Last-Click: 100% credit to the final click.
• First-Click: 100% credit to the initial click.
• Linear: Uniform credit distribution across all touchpoints.
• Data-Driven Attribution (DDA): Credit allocation learned via causal inference models (specifically CausalMTA).

Dataset Statistics:

• Source: Anonymized traffic logs from Taobao's advertising system.
• Scale: 79 million clicks, 15.1 million distinct items, and 0.8 million users.
• Features: Includes categorical features (user, item, context), behavior sequences (purchase history with visual similarity scores), and continuous attribution weights for all four mechanisms.
• Compliance: All personally identifiable information is irreversibly hashed.

2.2 The PyMAL Library

To facilitate reproducible research, the authors released PyMAL, an open-source Python library built on PyTorch. It implements a unified framework for:

• Single-Attribution Baselines: (e.g., BASE model).
• Multi-Task Learning (MTL) Models: Including Shared-Bottom, MMoE, PLE, and HoME.
• State-of-the-Art MAL Models: Including NATAL (kNowledge Aggregation with the carTesian AuxiLiary training).

2.3 Proposed Model: Mixture of Asymmetric Experts (MoAE)

Motivated by the analysis of existing baselines, the authors propose MoAE, a novel MAL architecture designed to satisfy two core principles:

1. Full Knowledge Acquisition: Learning multi-attribution knowledge comprehensively.
2. Main-Task-Prioritized Utilization: Leveraging this knowledge to maximize the primary task performance.

MoAE Architecture:

• MoE Backbone: Inspired by PLE, it employs a shared expert for public conversion patterns and attribution-specific experts for unique patterns under different mechanisms.
• Asymmetric Transfer Module: A main-task-centric feature transfer mechanism that aggregates auxiliary knowledge into the main task predictor in an asymmetric manner, ensuring the primary objective is prioritized.
• Predictors: Separate MLPs for each attribution mechanism.

3. Key Experimental Findings

The authors conducted comprehensive experiments on MAC using the PyMAL library, yielding three primary insights:

3.1 Generality of MAL

MAL consistently improves prediction accuracy across different target attribution mechanisms (Last-Click, First-Click, DDA, Linear).

• Significant Gains: The most notable improvements were observed for users with long and complex conversion paths. For example, under the Last-Click target, MoAE improved GAUC by +2.12 points over the single-attribution baseline.
• Noise Sensitivity: The First-Click setting yielded the smallest gains (0.34pt GAUC), attributed to the inherent noise in first-click labels due to long time gaps and confounding factors (e.g., subsequent promotions).

3.2 Impact of Auxiliary Objective Choice

The choice of auxiliary targets is critical and task-dependent:

• Scaling Effect: Generally, performance scales with the number of auxiliary targets, and incorporating Cartesian product auxiliary targets (CAT) yields further gains.
• First-Click Specificity: When predicting First-Click conversions, indiscriminate addition of auxiliary tasks hurts generalization. Only the Last-Click auxiliary target provided significant gains, likely because it offers the greatest temporal discrepancy and helps the model learn intrinsic intent more effectively.

3.3 Architectural Design Principles

The study identifies two paramount design principles for effective MAL:

1. Fully learning multi-attribution knowledge: Achieved via MoE structures (as seen in PLE/HoME).
2. Fully leveraging knowledge for the main task: Achieved via asymmetric knowledge transfer (as seen in NATAL).

• MoAE Performance: By combining both principles, MoAE outperformed all baselines, including the previous state-of-the-art NATAL, across all settings. It achieved a GAUC improvement of up to +0.39 points over NATAL in the Linear setting.

3.4 Source of Performance Gains

Ablation studies confirmed that performance gains stem from multi-attribution supervision signals rather than mere parameter scaling. When auxiliary task weights were set to zero (retaining the larger parameter count of MAL models), performance dropped to near-baseline levels.

4. Contributions and Significance

The paper claims the following contributions:

1. Data Resource: The release of MAC, the first multi-attribution CVR benchmark, filling a critical data gap and enabling systematic MAL research.
2. Software Toolkits: The release of PyMAL, an open-source library covering representative baselines and the proposed MoAE, ensuring fair and reproducible comparisons.
3. Actionable Insights: The identification of key design principles (MoE for knowledge acquisition + Asymmetric Transfer for task prioritization) and the nuanced understanding of auxiliary objective selection.
4. Effective Approach: The proposal of MoAE, which demonstrates substantial superiority over existing methods, raising the GAUC metric significantly across diverse attribution settings.

The authors conclude that the MAC benchmark, PyMAL toolkit, and the derived insights will serve as foundational resources to accelerate future innovation in multi-attribution learning and more holistic conversion modeling. They suggest future work could explore integrating stronger foundation model capabilities (e.g., chain-of-thought reasoning) and more comprehensive conversion objectives.