Towards Multi-Stakeholder Vulnerability Notifications in the Ad-Tech Supply Chain

1. Problem Statement

The online advertising supply chain is a complex, opaque ecosystem involving multiple stakeholders (publishers, ad-networks, and advertisers) with misaligned incentives. A prevalent form of ad fraud known as dark pooling allows low-quality (brand-unsafe) publishers to bundle their ad inventory with high-quality publishers using a shared "Seller ID." This makes the low-quality inventory indistinguishable to advertisers, leading to brand safety violations and funding misinformation sites.

Current mitigation strategies, such as industry standards (ads.txt, sellers.json, Supply Chain Object), suffer from low adoption and enforcement. Furthermore, existing vulnerability notification research primarily focuses on single-stakeholder scenarios (e.g., notifying a single website operator). There is a significant gap in understanding how to effectively notify and coordinate remediation across multi-stakeholder ecosystems where:

• Vulnerabilities exist across interdependent entities.
• Resolution requires coordinated action.
• Incentives are often conflicting (e.g., ad-networks may lose revenue by removing problematic inventory).

2. Methodology

The authors designed and executed a nine-month automated multi-stakeholder vulnerability notification pipeline to evaluate the effectiveness of academic and activist notifications in mitigating dark pooling.

A. Vulnerability Detection

• Static Analysis: Crawler collected ads.txt and sellers.json files from problematic publishers (categorized as misinformation, typosquatting, phishing, etc.) and top-1M websites. They identified "dark pools" where unrelated publishers shared a Seller ID on an ad-network.
• Dynamic Analysis: Used a headless browser (Selenium) to load problematic pages, capture HTTP archives (HAR), and verify that problematic publishers were actively monetizing inventory using the Seller IDs of reputable "victim" publishers.
• Dataset: Identified 26.1K dark pools involving 399 problematic publishers, 1.7K victim publishers, 962 ad-networks, and 635 advertisers.

B. Notification Campaign Design

The study utilized a sequential multi-round approach to avoid "contamination" (where one stakeholder's action influences another's response in a way that confounds attribution).

• Recipients: 2,900 unique entities were identified.
  • Round 1: 1,700 Victim Publishers.
  • Round 2: 644 Ad-networks.
  • Round 3: 635 Advertisers.
• Sender Reputation (Treatment Groups): Recipients were randomly split into three groups:
  1. Academics (T1): Notifications sent from university email addresses.
  2. Activists (T2): Notifications sent with branding from CheckMyAds (CMA), a known ad-tech watchdog (though sent via university servers to protect CMA's reputation).
  3. Control (C): No notification during the study (notified only at the end).
• Content: Emails provided entity-specific evidence (e.g., specific Seller IDs, screenshots for advertisers) and actionable remediation steps.

C. Evaluation Metrics

The authors used Propensity-Matched Difference-in-Differences (PM-DiD) to measure causal impact.

• Metrics:
  • Victim Publishers: Reduction in the number of problematic domains pooling their Seller ID (probdomainsvp).
  • Ad-networks: Reduction in Seller IDs they issued that were used in dark pools (poolsad) and those issued by partners (partnerpoolsad).
  • Advertisers: Reduction in dark pools associated with the ad-networks serving their ads.
• Analysis: Compared pre- and post-notification changes in treatment groups against matched control groups.

3. Key Contributions

1. First Multi-Stakeholder Notification Study: This is the first research to systematically evaluate vulnerability notifications in a fragmented, multi-stakeholder supply chain (ad-tech) where incentives are misaligned.
2. Operational Pipeline: Developed an automated framework for detecting dark pooling at scale and tailoring notifications to specific stakeholder roles (technical details for networks/publishers, brand safety impact for advertisers).
3. Sender Reputation Analysis: Investigated whether the source of the notification (Academic vs. Activist) impacts remediation rates in a complex ecosystem.
4. Stakeholder Influence Mapping: Quantified the relative influence of publishers, ad-networks, and advertisers in resolving supply chain vulnerabilities.

4. Key Results

A. Effectiveness of Notifications

Notifications are an effective method for reducing dark pooling.

• Overall Remediation: Between 54% and 82% of notified entities showed remedial action compared to controls.
• Ad-networks (Most Effective): 81.6% of notified ad-networks remediated their own pools, and 76.9% remediated pools in partner networks. They are the most capable stakeholders because they control Seller ID assignment.
• Publishers (Least Effective): Only 54% of notified publishers took remedial action. However, when they did act, the effect size was the largest (mean reduction of -30.2 problematic domains), likely because they often sit in very large pools.
• Advertisers: 72.6% showed mitigation, driven by their ability to pressure ad-networks.

B. Impact of Sender Reputation

• General Finding: For ad-networks and advertisers, there was no statistically significant difference in remediation rates between notifications from academics and activists.
• Exception (Publishers): Publishers were significantly more likely to remediate when notified by activists (CheckMyAds) than by academics.
  • Reasoning: Activists have a history of "naming and shaming" ad-tech entities on social media. Publishers, fearing reputational damage from activist exposure, responded more urgently to activist notifications.

C. Stakeholder Engagement (Qualitative)

• Themes: Responses fell into three categories: (1) Effort towards resolution (requesting details, fixing issues), (2) Demonstration of concern (fear of reputation loss), and (3) Lack of trust/resources (doubting data, phishing fears, lack of technical staff).
• Trust Issues: Some entities initially suspected phishing or lacked the technical resources to fix ads.txt files, highlighting a need for better industry education.

5. Significance and Implications

• Industry Solution: The paper proposes that industry bodies like the IAB Tech Lab (which already crawls ads.txt and sellers.json) should integrate this detection logic to create an automated, continuous fraud-prevention pipeline. Since ad-networks are the most influential stakeholders and are fewer in number than publishers, targeting them yields high impact.
• Research Framework: Provides a template for studying vulnerability notifications in other complex supply chains (e.g., software supply chains, fintech) where interdependencies and misaligned incentives exist.
• Strategic Ordering: The study suggests that in multi-stakeholder ecosystems, notifications should be ordered strategically (e.g., notifying the most capable entity first, or the most influential last) to build pressure and ensure attribution.
• Ethical Design: Demonstrates how to conduct large-scale security research ethically, including delayed notification for control groups and minimizing infrastructure stress.

Conclusion: The study proves that vulnerability notifications can effectively mitigate ad inventory fraud in a multi-stakeholder environment, provided the notifications are targeted at the most capable stakeholders (ad-networks) and tailored to the specific incentives and trust levels of the recipients.