Cyber Threat Intelligence for Artificial Intelligence Systems

This paper investigates the evolution of cyber threat intelligence to address AI-specific security threats by analyzing current gaps, proposing a structured knowledge base with concrete indicators of compromise across the AI supply chain, and outlining techniques for measuring artifact similarity to support a practical, AI-tailored defense framework.

The Gist

Imagine the world of cybersecurity as a massive, high-tech fortress. For decades, the guards (security experts) have been trained to spot intruders trying to break in through windows, pick locks, or sneak in through the front door. They use a giant "Wanted Poster" database (Cyber Threat Intelligence, or CTI) that lists the faces, fingerprints, and known tricks of thieves.

But now, the fortress has a new, very smart, but slightly naive employee: Artificial Intelligence (AI).

This paper, written by a team from Orange Innovation Poland, argues that our old "Wanted Posters" and security guards don't know how to catch thieves who are specifically trying to trick, poison, or steal this new AI employee. Here is the breakdown of their findings, translated into everyday language.

1. The Problem: The Old Rules Don't Fit the New Game

Think of traditional cybersecurity like a bouncer at a club. He checks IDs, looks for known troublemakers, and checks if you're carrying a weapon. It works great for humans.

But AI is different. It's not a person; it's a giant, invisible brain that learns by reading millions of books (data).

• The Old Threat: A thief breaks a window to steal a TV.
• The New AI Threat: A thief doesn't break the window. Instead, they sneak into the library and swap a few pages in the books the AI is reading. Now, the AI thinks "Fire" means "Water," or it believes that a picture of a stop sign is actually a speed limit sign.

The paper says our current security tools are looking for broken windows, but they are blind to the book page swaps. We need a new kind of "Wanted Poster" specifically for AI crimes.

2. The New "Wanted Posters" (Indicators of Compromise)

In the old days, a "clue" (Indicator of Compromise) was a specific file name or a bad IP address (like a bad phone number).

For AI, the clues are weirder. The paper suggests we need to look for:

• Poisoned Recipes: If a chef (the AI) is making soup, did someone sneak a little bit of poison into the ingredients before the cooking started? (This is called Data Poisoning).
• Fake Glasses: Did someone put special stickers on a stop sign that only the AI can see, making it think the sign is a "Go" signal? (This is an Adversarial Example).
• The "Jailbreak" Note: Did someone whisper a secret code to the AI that tricks it into ignoring its safety rules? (This is Prompt Injection).

The authors are trying to build a database that lists these specific "poisoned recipes" and "fake glasses" so security systems can spot them before the AI goes to work.

3. Where Do We Get the Clues? (The Sources)

The paper reviews where we can find information about these AI crimes. They found three main types of sources, like different sections of a library:

• The "Bug List" (Vulnerability Databases): Like a list of known weak spots in a car (e.g., "The brakes fail if it rains"). Examples: AVID and OWASP.
  • Verdict: Good, but often incomplete. They list the weak spots but don't always show how the bad guys actually used them.
• The "Crime Scene Reports" (Incident Databases): Like police reports detailing exactly what happened when a bank was robbed. Examples: AI Incident Database (AIID).
  • Verdict: Very useful! These tell real stories, like "An autonomous car hit a pedestrian because it confused a plastic bag for a rock." This helps us understand the consequences.
• The "Thief's Playbook" (Adversary Tactics): Like a manual written by the thieves explaining their tricks. The big one here is MITRE ATLAS.
  • Verdict: This is the gold standard. It maps out exactly how hackers attack AI, step-by-step, similar to how the famous MITRE ATT&CK framework maps out attacks on regular computers.

4. The Challenge: Finding a Needle in a Haystack

Here is the tricky part. If a hacker steals a model (the AI brain) and changes just one tiny line of code, it looks almost identical to the original. How do you catch them?

The paper suggests using Digital Fingerprints (Hashing).

• Analogy: Imagine you have a giant library of books. A thief steals a book, changes the font size, and adds a doodle on page 5. A normal search for the title won't find it.
• The Solution: The authors suggest using "Deep Hashing." This is like taking a photo of the entire vibe of the book. Even if the font changes, the "vibe" (the mathematical fingerprint) stays similar enough that the security system can say, "Hey, this looks 95% like that stolen book we know about!"

This allows security tools to catch modified AI models even if the hackers tried to disguise them.

5. Why This Matters (The Conclusion)

The authors conclude that we are currently flying blind. We have great tools for protecting regular computers, but we are struggling to protect AI.

• The Gap: We don't have enough "Wanted Posters" for AI-specific crimes.
• The Fix: We need to build a specialized intelligence system that understands AI's unique weaknesses (like poisoned data or prompt injections).
• The Goal: To create a system where, if a hacker tries to trick an AI, the security guard doesn't just say "Access Denied," but says, "I see you're trying to use a 'Poisoned Recipe' tactic. I've seen this before, and here is how we stop it."

In short: The paper is a call to action. We need to stop treating AI security like regular computer security. We need new maps, new fingerprints, and new rules of the road to keep our AI employees safe from the clever tricks of modern cybercriminals.

Technical Summary

Here is a detailed technical summary of the paper "Cyber Threat Intelligence for Artificial Intelligence Systems" by Krawczyk et al.

1. Problem Statement

As Artificial Intelligence (AI) systems become integral to critical infrastructure and daily products, they introduce unique attack surfaces that traditional Cyber Threat Intelligence (CTI) frameworks cannot effectively address.

• The Gap: Conventional CTI focuses on IT assets (networks, servers, software) and known vulnerabilities (e.g., buffer overflows, SQL injection). It lacks the taxonomy, indicators, and workflows to detect AI-specific threats such as data poisoning, model backdoors, adversarial examples, prompt injection, and model inversion attacks.
• The Challenge: AI attacks often occur across the entire ML lifecycle (development, training, deployment) rather than just at the network perimeter. Furthermore, existing resources for AI security are fragmented, inconsistent, or lack the structured data required for automated threat detection and response.
• Objective: The paper aims to define how CTI must evolve to protect AI systems, identifying the necessary components for an AI-oriented CTI knowledge base and the methods to query it.

2. Methodology

The authors conducted a Systematic Literature Review (SLR) following established guidelines for information systems research.

• Search Strategy: They queried academic databases (Google Scholar, Scopus) using keywords such as "Cyber threat intelligence," "CTI for AI," "AI incidents," "AI vulnerabilities," and "AI cyber threats."
• Analysis & Extraction: Papers were analyzed to extract information regarding CTI frameworks, data sources, Indicators of Compromise (IoCs), and applications in security tools.
• Synthesis: Insights were organized to answer four specific Research Questions (RQs):
  1. Differences between classical CTI and CTI for AI.
  2. Sources for building an AI CTI knowledge base and their reliability.
  3. Benefits of an AI CTI knowledge base for protection tools.
  4. Techniques for measuring similarity between collected IoCs and malicious AI artifacts.

3. Key Contributions

A. Comparative Analysis: Classical CTI vs. AI CTI

The paper establishes that AI CTI requires a fundamental shift in focus:

• Assets: Shift from servers/networks to training datasets, model weights, architectures, APIs, and inference pipelines.
• Vulnerabilities: Shift from standard software bugs to adversarial attacks, poisoning, and prompt injection.
• Lifecycle: AI attacks follow specific phases not present in traditional IT: Pre-attack (reconnaissance of ML artifacts), Training (poisoning), Model (backdoor insertion), and Deployment (evasion/inference manipulation).

B. Taxonomy of Data Sources

The authors categorize and evaluate existing resources for building an AI CTI knowledge base:

1. Vulnerability-Oriented Sources:
   • AVID (AI Vulnerability Database): An open-source repository for AI/ML vulnerabilities. It offers a structured taxonomy (Effect view: Security/Ethics/Performance; Lifecycle view: CRISP-DM stages). Status: Promising but needs more contributors.
   • OWASP, ENISA, SAIF: Provide guidelines and best practices but are not standalone vulnerability repositories.
2. Incident-Oriented Sources:
   • AI Incident Database (AIID): A community-driven repository of real-world AI harms. It uses core metadata (ID, description, harmed parties) and optional taxonomies (CSET, GMF). Status: Mature and reliable.
   • CSET AI Harm Taxonomy & GMF: Frameworks for classifying the consequences and technical causes of AI failures.
3. Adversary-Oriented Sources:
   • MITRE ATLAS: The AI equivalent of MITRE ATT&CK. It maps adversarial tactics, techniques, and procedures (TTPs) specific to AI (e.g., model extraction, evasion).
   • Specialized Datasets: Includes Prompt Injection datasets (e.g., Qualifire, Prompt Injection Attack Dataset) and Malicious Model repositories (e.g., malicious files on Hugging Face identified by ReversingLabs and NSFOCUS).

C. Defining AI-Specific Indicators of Compromise (IoCs)

The paper proposes that IoCs for AI must extend beyond file hashes and IPs to include:

• Model Artifacts: Suspicious model weights, modified architecture patterns, and tokenizer fingerprints.
• Data Artifacts: Unusual patterns in training datasets or poisoned data signatures.
• Behavioral Artifacts: Prompt injection patterns, jailbreak attempts, and anomalous inference outputs.

D. Similarity Measurement and Querying Techniques

To handle the complexity of AI assets, the paper reviews advanced hashing and similarity techniques:

• Deep Hashing: Transforms complex AI assets (weights, embeddings) into compact binary fingerprints to preserve semantic similarity. Allows for fast comparison without processing full data.
• Fuzzy Hashing & Similarity Digests: Algorithms like TLSH and LZJD (adapted from malware analysis) generate similarity scores to detect modified or polymorphic models.
• Semantic Consistency Hashing (SCH): A newer method that converts local similarity structures into probability distributions to maintain stable hash codes under perturbations.
• Application: These methods enable fast retrieval of exact or near-matches in large knowledge bases, allowing security tools to detect previously unseen but similar malicious models.

4. Results & Findings

• Current State: Existing resources (MITRE ATLAS, AIID) are the most mature and reliable, but the overall landscape is fragmented. Many specialized datasets suffer from labeling inconsistencies or limited scope.
• Gap Identification: There is a lack of standardized IoCs for AI. Current CTI tools cannot effectively scan for "poisoned" training data or "backdoored" model weights without custom integration.
• Feasibility: It is technically feasible to build an AI CTI system by integrating structured taxonomies (AVID, CSET) with similarity hashing algorithms (Deep Hashing, TLSH).
• Tool Integration: An AI CTI knowledge base would allow security tools to:
  • Scan models before deployment against known malicious signatures.
  • Correlate observed behaviors with historical incidents (via AIID).
  • Monitor for specific TTPs (via MITRE ATLAS) like data poisoning or prompt injection.

5. Significance

This paper provides a foundational roadmap for the next generation of cybersecurity in the AI era.

• Strategic Value: It moves the industry from reactive incident reporting to proactive threat intelligence, enabling organizations to anticipate AI-specific attacks.
• Technical Innovation: By proposing the adaptation of deep hashing and semantic similarity for AI assets, the authors offer a practical solution to the scalability problem of comparing large models and datasets.
• Standardization: The call for a unified framework for AI IoCs and the evaluation of existing taxonomies (AVID, MITRE ATLAS) serves as a blueprint for future standardization efforts in AI security.
• Future Research: The paper identifies the need for novel IoCs tailored to AI tampering (e.g., detecting subtle weight modifications) and the development of automated frameworks to monitor AI supply chains.

In conclusion, the authors argue that without a specialized CTI framework, organizations remain blind to the unique risks posed by AI systems. The proposed approach integrates traditional threat intelligence methodologies with AI-specific data structures and similarity algorithms to create a robust defense mechanism for the AI supply chain.