💻 computer science

NDN-Aware Cooperative Multi-Agent Deep Reinforcement Learning for Distributed Cache Pollution Attack Mitigation

This paper proposes an NDN-aware cooperative Multi-Agent Deep Deterministic Policy Gradient (MADDPG) framework that leverages decentralized learning and specialized state representations to effectively mitigate Cache Pollution Attacks in Named Data Networking, demonstrating superior cache efficiency, faster convergence, and enhanced resilience compared to single-agent and conventional approaches.

Original authors: Sethu S, Manikandan A

Published 2026-07-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sethu S, Manikandan A

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: A Library Under Siege

Imagine the internet not as a series of computers talking to each other, but as a massive, distributed library system called Named Data Networking (NDN). In this library, instead of asking for a specific computer's address, you ask for a specific book by its title.

The magic of this system is in-network caching. Think of every router (the traffic cops of the internet) as a small branch library. If a popular book is requested often, the branch keeps a copy on its shelf so the next person doesn't have to wait for it to be shipped from the main warehouse. This makes everything faster and smoother.

The Problem: The "Junk Mail" Attack
However, there is a villain in this story: the Cache Pollution Attack (CPA). Imagine a group of pranksters who know exactly how the library works. Instead of asking for the popular books, they start screaming for thousands of obscure, useless, or fake titles.

They flood the branch libraries with requests for "How to tie a shoelace with a rubber band" or "The history of invisible ink."
The library shelves get so full of this junk that the real, popular books get kicked off the shelves to make room.
When a normal person asks for a popular book, the library has to say, "Sorry, we don't have it," and they have to wait for it to be shipped from far away. The system slows down, and the user experience crashes.

The Old Solutions: Rigid Rules vs. One Smart Manager

Traditionally, libraries tried to stop this with static rules (e.g., "If a book is requested more than 10 times, keep it"). But smart attackers can easily trick these rigid rules.

Others tried using one single smart manager (Single-Agent AI) to watch the whole library and decide what to keep. But in a huge, distributed network, one manager can't see everything happening at once. By the time the manager notices the attack, it's too late, and the decision-making is too slow.

The New Solution: A Team of Smart Librarians (MADDPG)

This paper proposes a new way to defend the library using Multi-Agent Deep Reinforcement Learning (MADDPG).

The Analogy: A Team of Detective Librarians
Instead of one boss watching everything, imagine every branch library has its own smart, adaptive librarian (an AI agent).

They Watch Locally: Each librarian watches their own shelf. They see what people are asking for, how full the shelf is, and if someone is suddenly asking for weird, useless items.
They Learn Together (Cooperatively): Even though they make their own decisions, they are trained together. It's like a team of detectives who share their "case files" during training but solve crimes independently on the streets.
The Strategy:
- If a librarian sees a sudden flood of requests for "invisible ink," they realize, "This isn't normal popularity; this is an attack!"
- They immediately stop putting those junk books on the shelf.
- They kick out the old, low-value books to make room for the real popular ones.
- They might even ignore certain requests entirely to save space.

How They Tested It

The researchers built a digital simulation of this library system using a tool called ndnSIM. They didn't just guess; they used real-world attack data (from the CICIDS2017 dataset) to simulate a massive, coordinated attack where hackers tried to flood the system from multiple directions at once.

They compared three teams:

The Old Guard: Libraries using standard rules (like "keep the most recent books").
The Solo Detective: A library with one smart AI manager (Single-Agent DDQN).
The Detective Squad: The new team of cooperative librarians (Multi-Agent MADDPG).

The Results: The Squad Wins

The results were clear, and the "Detective Squad" (MADDPG) crushed the competition:

Better Shelf Space: The cooperative team kept the "hit ratio" (how often they had the right book ready) 18–22% higher than the single manager or the old rules.
Faster Service: Because the shelves weren't clogged with junk, people got their books much faster. The "interest satisfaction delay" (waiting time) was significantly lower.
Smarter Adaptation: When the attackers changed their tactics, the team of librarians adapted instantly. The single manager was slower to catch on.
Stability: The team's performance was smooth and consistent, whereas the single manager's performance jumped around wildly when the attack got intense.

Why This Matters (According to the Paper)

The paper concludes that for a distributed system like this, cooperation is key. You can't rely on one central brain to protect a network that is everywhere at once. By giving every router its own "brain" that learns to work with its neighbors, the network becomes much harder to trick and much faster for everyone.

In short: The paper shows that a team of smart, cooperating AI librarians is much better at keeping a library clean and organized during a chaotic attack than a single boss or a set of rigid rules.

Technical Summary: NDN-Aware Cooperative Multi-Agent Deep Reinforcement Learning for Distributed Cache Pollution Attack Mitigation

1. Problem Statement

Named Data Networking (NDN) enhances content delivery efficiency through in-network caching but introduces significant vulnerability to Cache Pollution Attacks (CPAs). In these attacks, adversaries flood the network with requests for low-value or malicious content to displace legitimate data, thereby degrading cache hit ratios, increasing retrieval latency, and compromising Quality of Service (QoS).

Existing mitigation strategies rely heavily on static thresholds, heuristic policies (e.g., LRU, LFU), or centralized single-agent learning models. These approaches face critical limitations in dynamic, distributed NDN environments:

Scalability Issues: Centralized or single-agent models struggle to react to coordinated, distributed attacks originating from multiple ingress routers.
Adaptability: Static heuristics fail to distinguish between legitimate popularity bursts and malicious traffic patterns.
Coordination Gaps: Single-agent learning lacks the global situational awareness required to mitigate attacks that propagate across a network of cooperative routers.

2. Methodology

The paper proposes an NDN-aware cooperative Multi-Agent Deep Deterministic Policy Gradient (MADDPG) framework designed for distributed CPA mitigation. The system models each NDN router as an independent intelligent agent that collaborates to optimize cache management without a central controller during execution.

Core Architectural Components

Centralized Training, Decentralized Execution (CTDE): The framework utilizes a CTDE paradigm. During training, a centralized critic aggregates state and reward information from all agents to learn a global policy. During execution, each router acts independently using its local actor network, ensuring scalability and fault tolerance.
NDN-Aware State Representation: Unlike generic traffic analysis, the state vector is explicitly designed around NDN architecture, incorporating:
- Pending Interest Table (PIT) occupancy dynamics.
- Cache popularity locality and entry aging behavior.
- Interest satisfaction delay.
- Attack-driven traffic fluctuations.
Action Space: Agents perform adaptive decisions including cache admission control, priority-based eviction, and selective bypassing of suspicious requests.
Reward Engineering: A CPA-sensitive reward function balances cache efficiency (maximizing hit ratio) with security goals (penalizing pollution and excessive misses). The reward is defined as $R_t = \alpha \cdot \Delta CHR_t - \beta \cdot CMR_t - \gamma \cdot P_t$ , where $CHR$ is the cache hit ratio, $CMR$ is the miss ratio, and $P_t$ represents polluted entries.

Comparative Baseline

The study conducts a direct comparative analysis between the proposed Multi-Agent DRL (MADDPG) and a Single-Agent DRL (DDQN) model under identical CPA scenarios. Both are evaluated against traditional baselines (LRU, LFU, Random) and other RL-based frameworks (ELDA, Q-ICAN).

Experimental Setup

Simulation Environment: The experiments were conducted using ndnSIM integrated with the NS-3 network simulator.
Dataset: Realistic benign and attack traffic patterns were modeled using the CICIDS2017 dataset. Adversarial behaviors (cache flooding, poisoning, interest flooding) were transformed into NDN-compatible interest request patterns.
Topology: A peer-to-peer and hierarchical topology of NDN routers was simulated, with each router acting as a learning agent in the multi-agent scenario.

3. Key Contributions

The paper outlines five primary contributions:

Framework Proposal: An NDN-aware cooperative Multi-Agent DRL framework using MADDPG with CTDE for distributed CPA mitigation.
Specialized State Representation: The design of a CPA-sensitive state vector integrating PIT occupancy, cache aging, and interest satisfaction dynamics specifically for NDN.
Distributed Coordination: A mechanism enabling intelligent, decentralized cache admission and eviction decisions across multiple routers under coordinated attack conditions.
Comparative Analysis: A rigorous comparison between Single-Agent (DDQN) and Multi-Agent (MADDPG) approaches, demonstrating the superiority of MARL in scalability and convergence.
Statistical Validation: Extensive validation including ANOVA analysis, confidence interval evaluation, and ablation studies to verify learning stability and robustness.

4. Results and Performance Evaluation

Experimental results, validated through statistical analysis (p < 0.01), demonstrate that the MADDPG framework consistently outperforms both the DDQN baseline and traditional caching policies.

Cache Hit Ratio: The proposed MADDPG framework achieved approximately 18–22% higher cache hit ratios compared to DDQN and conventional approaches under coordinated CPA scenarios. In specific comparisons against single-agent baselines, the gain reached up to 35% in P2P-like topologies.
Interest Satisfaction Delay: MADDPG significantly reduced interest satisfaction delay compared to DDQN and static heuristics, indicating faster content retrieval and reduced upstream bandwidth usage.
Convergence and Stability: The MADDPG reward trajectory showed smoother convergence and lower variance than DDQN, attributed to the CTDE paradigm mitigating non-stationarity in multi-agent systems.
Ablation Study: Removing key components (PIT-aware features, cooperative agents, or adaptive rewards) resulted in significant performance degradation. Notably, the absence of cooperative agents caused the largest drop in performance, highlighting the necessity of distributed coordination.

5. Significance and Claims

The paper positions this work as one of the first NDN-aware cooperative MARL frameworks specifically designed for distributed CPA mitigation using CTDE-enabled adaptive cache intelligence.

Scalability: By shifting from centralized or single-agent learning to a decentralized multi-agent approach, the framework addresses the scalability limitations of previous methods, making it suitable for large-scale NDN deployments such as ISP backbones and blockchain overlays.
Resilience: The system demonstrates superior resilience against coordinated and dynamic attack strategies that typically defeat static threshold-based or single-agent detection systems.
Architectural Alignment: The methodology aligns closely with NDN's inherent distributed nature, leveraging local observations to achieve global optimization without the overhead of global state synchronization.

The authors conclude that while the current implementation relies on simulation and public datasets, the proposed framework offers a robust, scalable, and adaptive solution for securing next-generation NDN environments against sophisticated cache pollution threats. Future work is identified as focusing on Graph Neural Networks (GNNs) for topological dependencies, federated learning, and validation with large-scale real-world traffic traces.