💬 NLP

Assessing Deanonymization Risks with Stylometry-Assisted LLM Agent

This paper introduces SALA, a stylometry-assisted LLM agent framework that evaluates authorship deanonymization risks in news articles and proposes a guided recomposition strategy to effectively mitigate these risks while preserving textual meaning.

Original authors: Boyang Zhang, Yang Zhang

Published 2026-02-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Boyang Zhang, Yang Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are writing a secret letter to a friend, but you want to make sure no one can guess who wrote it. You think you've done a good job by removing your name. But what if a super-smart detective could look at your handwriting, the way you use commas, and your favorite words to figure out exactly who you are?

That is the scary reality this paper explores. The authors built a "digital detective" powered by Artificial Intelligence (AI) to see just how easy it is to unmask anonymous writers in news articles. But instead of just being a villain, this detective also acts as a bodyguard, teaching writers how to hide better.

Here is the breakdown of their work, using some everyday analogies:

1. The Problem: The "Digital Fingerprint"

In the past, if you wanted to hide your identity, you just took off your name. But people have "digital fingerprints." Just like you might always wear a red hat or hum a specific tune, writers have habits. They might always use long sentences, love specific words, or have a unique way of starting paragraphs.

The authors found that modern AI (Large Language Models) is getting so good at reading that it can spot these habits instantly. It's like a detective who doesn't need a name tag; they just look at your shoes and say, "Ah, that's definitely Bob."

2. The Solution: The "SALA" Detective

The team built a special AI agent called SALA (Stylometry-Assisted LLM Analysis). Think of SALA as a detective with two superpowers working together:

The Calculator (Stylometry): This part counts things. It measures the average length of sentences, counts how many big words are used, and checks punctuation. It's like a forensic scientist measuring the exact size of a footprint.
The Intuitionist (LLM Reasoning): This is the AI's brain. It reads the text and says, "This feels like a serious, formal writer," or "This sounds like someone who likes sarcasm."

By combining the hard numbers (the footprint size) with the gut feeling (the vibe of the text), SALA becomes incredibly accurate at guessing who wrote a piece of text.

3. The "Library" Trick (The Database)

At first, the detective had to search the whole internet for every single clue, which was slow and often failed.

The authors added a Database Module. Imagine this as giving the detective a pre-organized library of famous writers. Instead of searching the whole world, the detective just checks the library.

Without the library: The detective guesses randomly.
With the library: The detective has a "Wanted" poster for thousands of authors. If the article matches the style of "Author X" in the library, the detective can say, "I'm 90% sure this is them!"

This made the detective terrifyingly good at finding the truth.

4. The Twist: The Detective Becomes a Bodyguard

Here is the most important part. The authors didn't just build a tool to break anonymity; they built a tool to fix it.

Once the AI detective figures out how it identified the author (e.g., "Oh, I knew it was Bob because he always uses the word 'furthermore' and writes very long sentences"), it can give Bob a shield.

The "Guided Recomposition": The AI tells the writer: "Hey, you're too easy to spot. You use too many long words. Let's rewrite this. Use shorter sentences and swap 'furthermore' for 'also'."
The Result: The article still means the same thing, but the "digital fingerprint" is wiped clean. It's like Bob putting on a disguise that changes his walk and voice so the detective can't recognize him anymore.

5. Why This Matters

This paper is a wake-up call.

The Bad News: If you think you are anonymous online, you might not be. A smart AI can likely figure out who you are just by reading your writing style.
The Good News: We now have a way to test our own privacy. We can use this "detective" to see if our writing is safe, and if it's not, we can use the "bodyguard" features to rewrite it and protect ourselves.

In a nutshell: The authors built a super-smart AI that can unmask anonymous writers by looking at their writing habits. But they also showed us how to use that same AI to "scramble" those habits, helping writers stay safe in a world where AI is getting better at reading between the lines.

1. Problem Statement

The rapid advancement of Large Language Models (LLMs) and LLM-based agents has created a significant privacy threat: unintended deanonymization. While traditional authorship attribution relied on supervised models with handcrafted features, modern LLM agents can implicitly encode authorial styles, perform zero-shot/few-shot identification, and utilize external tools (like web search) to link anonymous text to specific authors.

This poses severe risks for individuals relying on anonymity, such as journalists, whistleblowers, and researchers in sensitive fields. The paper addresses two critical gaps:

Threat Assessment: There is a lack of frameworks to quantify how effectively LLM agents can deanonymize text in realistic scenarios (both targeted and open-world).
Mitigation: Existing defenses are often generic; there is a need for proactive, interpretable strategies that reduce authorship identifiability while preserving the semantic utility of the text.

2. Methodology: The SALA Framework

The authors propose a modular LLM agent framework designed to act as both an adversary (to assess risk) and an advisor (to mitigate risk). The system operates through a four-stage pipeline augmented by two key modules.

A. The Four-Stage Pipeline

Information Extraction: The agent analyzes the target article to extract metadata (publication date, topic, geo-location) to narrow the search scope.
Candidate Search: The agent retrieves a list of potential authors.
- Zero-Shot: Uses web search APIs and internal LLM knowledge.
- Database-Augmented: Queries a pre-constructed database of author profiles and writing samples for higher efficiency and accuracy.
Candidate Matching (Core Innovation): The agent performs pairwise matching between the target article and candidate profiles using SALA (Stylometry-Assisted LLM Analysis).
- Hybrid Approach: SALA integrates quantitative stylometric features (computed via Python/NLP tools like NLTK) with qualitative LLM reasoning.
- Feature Categories: Lexical (e.g., unique word count, TTR), Syntactic (e.g., sentence length, POS variation), Readability, Semantic (polarity, subjectivity), and Overall Style.
- Mechanism: The agent is prompted to compare specific feature values (e.g., "Article A has avg word length $X$ , Article B has $Y$ ") to determine the likelihood of shared authorship. This grounds the LLM's reasoning in explicit metrics, reducing hallucinations common in purely qualitative analysis.
Result Reflection: The agent interprets the matching results to identify which specific stylistic features contributed most to the identification, forming the basis for mitigation.

B. Supporting Modules

Database Module: Pre-aggregates author profiles and sample writings. This allows the agent to bypass costly web searches, use embedding similarity for candidate retrieval, and compare the target against aggregated, stable author profiles rather than single samples.
Anonymization Enhancement Module: Uses the "Result Reflection" insights to generate guided rewriting prompts. Instead of generic paraphrasing, it targets specific risk factors (e.g., "diversify sentence length," "replace domain-specific jargon") to reduce identifiability.

3. Key Contributions

SALA Method: A novel authorship attribution strategy that combines deterministic stylometric metrics with LLM reasoning, offering higher precision and interpretability than pure LLM analysis (LDA) or embedding similarity (ES).
Dual-Purpose Agent Framework: A unified system that simulates adversarial deanonymization attacks and simultaneously provides evidence-based defensive recommendations.
Privacy Evaluation Framework: A structured approach to quantify authorship risk in both Targeted Attacks (verifying if a text belongs to specific known authors) and Open-World Attacks (identifying the author from a vast pool without prior targets).
Guided Recomposition Strategy: A defense mechanism that leverages the agent's reasoning trace to generate targeted rewriting instructions, effectively lowering re-identification rates while maintaining content utility.

4. Experimental Results

The framework was evaluated on large-scale news datasets (All the News 2.0 and CrossNews) containing over 2.5 million articles.

Attack Performance (Risk Assessment)

Candidate Search:
- Zero-Shot: Poor performance. With 20 candidates, the true author was found only 1.2% of the time.
- Database-Augmented: Significant improvement. With 20 candidates, coverage rose to 78.1%, reaching 93.1% with 100 candidates.
Matching Accuracy (Targeted Attack):
- SALA significantly outperformed baselines. With 500 samples per candidate, SALA achieved an F1-score of 0.827, compared to 0.699 for LLM Direct Analysis (LDA) and 0.500 for Embedding Similarity (ES).
- Performance plateaued around 300 samples, indicating diminishing returns beyond that point.
Open-World Attack:
- Without a database, the system failed to make correct predictions (0.000 accuracy).
- With the database and SALA, the system achieved a Top-3 accuracy of 22.4% (Filtered), demonstrating that even in open-world settings, LLM agents pose a non-trivial deanonymization threat.

Mitigation Performance (Defense)

Guided Recomposition vs. Direct Paraphrasing:
- Direct Paraphrasing: Reduced targeted attack success from 0.827 to 0.762 (ineffective).
- Guided Recomposition: Reduced targeted attack success to 0.561 (near random) and open-world success to 0.012.
- Conclusion: Targeted, evidence-based rewriting is far more effective at breaking authorship signals than generic paraphrasing.

5. Significance and Implications

Dual-Edged Sword: The paper highlights that LLM agents are powerful tools for both privacy violation and privacy protection. The same reasoning capabilities that allow an agent to deanonymize a text can be used to guide the anonymization of that text.
Interpretability: By grounding LLM reasoning in quantitative stylometric features, the framework makes the "black box" of authorship attribution transparent, allowing users to understand why they are identifiable.
Proactive Defense: The study moves beyond theoretical risk assessment to practical defense, demonstrating that specific, guided modifications to text can effectively neutralize LLM-based deanonymization attacks.
Future Directions: The authors note limitations regarding language diversity (English-only) and the need for balanced author profiles in databases to avoid bias, suggesting future work in multilingual and diverse domain applications.

In summary, this work provides a critical blueprint for understanding the privacy vulnerabilities introduced by autonomous AI agents and offers a robust, interpretable framework for mitigating these risks through stylometry-assisted analysis and guided text recomposition.