"Abuse Risks are Often Inherent to Product Features": Exploring AI Vendors' Bug Bounty and Responsible Disclosure Policies

Technical Summary: "Abuse Risks are Often Inherent to Product Features": Exploring AI Vendors' Bug Bounty and Responsible Disclosure Policies

Problem Statement

As artificial intelligence (AI) and large language models (LLMs) become deeply embedded in production software, the security implications of these probabilistic systems remain unclear. Unlike traditional deterministic software, AI systems operate on rules learned from data, leading to unique vulnerabilities such as prompt injection, model extraction, and hallucinations. A critical gap exists in understanding whether vendors have adapted their vulnerability disclosure policies (VDPs) and bug bounty programs (BBPs) to address these specific risks. Prior empirical studies on disclosure policies predate the rapid adoption of AI, and it is unknown if existing mechanisms effectively incentivize the reporting of AI-specific flaws or if vendors recognize them as distinct from traditional security violations.

Methodology

The authors conducted a mixed-methods study analyzing the vulnerability disclosure policies of 264 AI vendors. The research employed three primary data collection and analysis strategies:

1. Vendor Identification and Policy Collection:

   • Vendors were identified using Gartner's product lists across nine AI categories, resulting in a unique list of 264 companies.
   • Disclosure policies were collected via security.txt files, major bug bounty platforms (HackerOne, Bugcrowd, Huntr), and manual web searches.
   • The final corpus included 168 policy documents (BBPs, VDPs, or security contacts).

2. Qualitative and Longitudinal Policy Analysis (RQ1 & RQ2):

   • High-Level Structure: Policies were mapped against an expanded 15-element framework (based on Walshe et al. [79]), including new elements like Service-Commitment and Contact-Channel to capture AI-specific nuances.
   • AI-Specific Analysis: A subset of 48 vendors with explicit AI mentions underwent snapshot and longitudinal analysis. Researchers extracted 422 thematic units and 205 historical policy changes (2018–2025) to track evolution.
   • Coding Reliability: Two authors independently coded 100 units, achieving a Cohen's Kappa of 0.816.

3. Alignment Analysis (RQ3):

   • Incident Data: 320 real-world AI incidents were extracted from the AI Incident Database (AIID) and mapped to the MIT AI Risk Taxonomy.
   • Academic Research: 359 AI security papers from four major security conferences (2020–2024) were analyzed to create a meta-taxonomy of research focus.
   • Comparison: The study compared the timing and thematic focus of vendor policies against these incident and academic datasets.

Key Contributions

• Systematic Policy Audit: The first large-scale empirical study (264 vendors) characterizing the state of AI vulnerability disclosure policies.
• Taxonomy of Vendor Approaches: The identification of three distinct vendor profiles based on their stance toward AI vulnerabilities: Proactive Clarification, Silent, and Restrictive.
• Longitudinal Tracking: A detailed timeline of policy evolution, revealing when and how vendors began addressing AI-specific risks.
• Alignment Gap Analysis: A comparative assessment showing the misalignment between vendor disclosure scopes, real-world incidents, and academic research priorities.

Key Results

1. State of Disclosure (RQ1)

• Adoption: Only 64% of AI vendors have a public disclosure channel. 36% have no established policy or contact mechanism.
• Explicit AI Recognition: Only 18% of vendors explicitly mention AI risks in their disclosure documents. The first mention occurred in 2018.
• Scope Consistency:
  • In-Scope: Vulnerabilities defined by traditional security goals (e.g., system integrity, data access, authorization) are most consistently accepted (91%). Specific AI model vulnerabilities like prompt injection (71%), adversarial examples (80%), and model extraction (90%) are frequently in scope.
  • Out-of-Scope: Issues related to AI features and content safety are rarely accepted. Jailbreaking (27%), harmful output (45%), and hallucination (17%) are most commonly declared out-of-scope.
• Rewards: Among vendors offering rewards for AI issues, some assign lower payouts (perceived lower impact), while others assign higher payouts (treating models as core assets). However, no AI-specific reward exceeded the top-tier bounties for traditional critical vulnerabilities.

2. Vendor Profiles (RQ2)
The study classifies vendors into three profiles:

• Proactive Clarification (17%, n=46): Typically model providers (e.g., Google, Microsoft, Meta). They explicitly clarify in/out-of-scope AI reports, often providing separate channels, severity frameworks, and testing guides.
• Silent (44%, n=115): Typically Gen-AI apps. They do not mention AI reports in their policies. While they may accept reports under generic terms, they offer no guidance or assurance.
• Restrictive (39%, n=103): Often data science providers or companies with no disclosure channel. They either explicitly reject AI reports (citing "inherent abuse risks" or lack of standalone vulnerability) or lack any mechanism to receive them.

3. Alignment with Incidents and Research (RQ3)

• Timing Lag: Vendors addressed AI vulnerabilities in their policies significantly later than academic research and real-world incidents. For example, while model extraction attacks were discussed academically in 2016, major vendors only clarified in-scope status in 2023.
• Thematic Misalignment:
  • Vendors & Academia: Focus heavily on upstream technical vulnerabilities (model extraction, adversarial examples, prompt injection).
  • Real-World Incidents: Dominated by downstream content safety issues (discrimination, toxicity, misinformation), which account for over 40% of public incidents but are rarely in-scope for vendor bug bounties.
• Growth: All three datasets (policies, incidents, papers) show "hockey-stick" growth in the last few years, with policy mentions growing the fastest (70% of mentions in 2023–2024).

Significance and Claims

The paper claims that the current landscape of AI vulnerability disclosure is fragmented and reactive. The authors argue that while vendors are beginning to address technical AI flaws (like model extraction), they largely reject "abuse risks" (like hallucinations or harmful output) by framing them as inherent product features rather than fixable vulnerabilities.

The study highlights a critical disconnect: Academic research and real-world harm focus heavily on content safety and societal impact, whereas industry disclosure policies remain anchored in traditional security goals (confidentiality, integrity, availability). The authors suggest that vendors may be adopting a "ship it and patch later" approach similar to early software security, relying on bolt-on filters rather than secure-by-design AI. They conclude that without clearer policies and recognition of content safety as a security issue, the industry may fail to incentivize the discovery and remediation of the most prevalent AI risks.