Out of Sight: Compression-Aware Content Protection against Agentic Crawlers

Technical Summary: CAPE (Compression-Aware Protective Evolution)

Problem Statement
The proliferation of Large Language Model (LLM) agents equipped with reasoning, summarization, and memory capabilities has created a novel threat surface for online content. Unlike traditional scrapers that store pages verbatim, agentic crawlers actively retrieve, compress, and condense content to fit context budgets before reusing it in downstream tasks. Existing defenses are insufficient: access controls (e.g., robots.txt) are easily circumvented by agents mimicking standard browsers, and injection-based defenses (e.g., prompt injection) often degrade human readability or fail to protect the underlying content integrity. Furthermore, post-hoc watermarking and training-time poisoning cannot intervene during the inference-time ingestion phase where agents process content.

The paper identifies context compression as a critical, overlooked defense layer. Since agents must inevitably compress raw text to fit context windows, intervening at this stage offers an orthogonal defense mechanism that activates even after perimeter defenses are bypassed.

Methodology: CAPE Framework
The authors propose CAPE, a framework that protects high-value textual content by injecting invisible perturbations (using zero-width characters and other non-rendering Unicode tokens) into the source text. These perturbations are designed to be imperceptible to human readers but cause severe semantic and structural degradation when the text is processed by an agent's compression module.

CAPE formulates perturbation design as a constrained multi-objective optimization problem: maximize information loss during compression while minimizing the human-visible surface difference (HVID). The framework operates in three stages:

1. Structural Prior Discovery:

   • Utilizes an accessible surrogate compressor (white-box access) to optimize perturbations that disrupt the model's generation distribution.
   • Instead of targeting a specific output string, the objective maximizes entropy, shifts probability mass toward anomalous token regions, and suppresses fluent natural-language continuations.
   • High-scoring candidates are analyzed to extract structural priors: local degradation-associated fragments, fragment co-occurrence patterns, and position-length compatibility attributes. These form a "seed corpus."

2. Prior-Guided Evolutionary Adaptation:

   • Adapts the seed corpus to the target (black-box) compressor without gradient access.
   • Uses a genetic search approach where recombination and mutation are guided by the extracted structural priors rather than arbitrary character edits.
   • Employs a regularized fitness function that balances observed target degradation, consistency with surrogate-derived priors, feedback stability, and exploration memory (tabu search) to avoid local optima.
   • A dynamic annealing mechanism adjusts acceptance temperatures based on population diversity and novelty to balance exploitation and exploration.

3. Preference-Calibrated Query Selection:

   • Addresses the high cost of querying closed-source target compressors.
   • Trains a local preference ranker using pairwise comparisons of target feedback to predict candidate degradation potential.
   • Allocates the limited query budget using an acquisition function that balances predicted effectiveness, ranker uncertainty, and structural novelty, ensuring efficient exploration of the perturbation space.

Key Contributions

1. Defense Layer Identification: The paper is the first to identify context compression as a viable defense layer for content protection in the agentic era, establishing a content-layer defense that complements existing access-control methods.
2. CAPE Framework: The authors formulate the problem as a multi-objective optimization and propose CAPE, a unified framework combining distributional prior discovery, prior-guided evolution, and preference-calibrated query selection.
3. Empirical Validation: Systematic experiments demonstrate that CAPE significantly outperforms baselines across three content types (long-form text, code, dialogue) and four target settings (GPT-4.1, Gemini 3 Flash, LangGraph, GitHub Copilot).
4. Open Release: The prototype and evaluation materials are publicly released to support reproducibility.

Experimental Results

• Effectiveness: On closed-source compressors, CAPE improves information degradation by up to 75.8% over the strongest baseline. In real-world agent workflows, it induces a 59.7% accuracy drop in downstream tasks (LangGraph) and significantly increases malformed output rates in GitHub Copilot.
• Stealth: CAPE maintains a human-visible difference (HVID) of approximately 1.4%, rendering the protected content visually indistinguishable from the original.
• Robustness: The method transfers effectively across different compression paradigms (abstractive rewriting, hard token selection, soft representation compression) and content types.
• Ablation: Removing any of the three core stages (prior discovery, guided evolution, or query selection) leads to significant performance degradation, confirming the necessity of the full pipeline.

Significance and Claims
The paper claims to reveal context compression as a new, practical defense layer for the agent era. It argues that as agents increasingly rely on compressed representations for memory and reasoning, protecting the fidelity of this compression process is essential for content owners. CAPE offers a proactive, stealthy mechanism that does not rely on platform-level controls or voluntary compliance. The authors position this work as a foundational step toward content protection in the agent era, motivating future research into compression-aware defenses while acknowledging limitations regarding adaptive adversaries (e.g., those who strip invisible characters) and the rapid evolution of agent ecosystems.