🤖 machine learning

DeepInvert: Semi-Supervised Embedding Inversion Against Obfuscated Language Models

This paper introduces DeepInvert, a semi-supervised embedding inversion attack that exploits the preserved semantic structure in obfuscated language model representations to recover original tokens with significantly higher accuracy than prior methods, revealing that current obfuscation defenses often fail to balance privacy protection with task utility.

Original authors: Zhicong Huang, Cheng Hong, Tao Wei

Published 2026-08-06

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhicong Huang, Cheng Hong, Tao Wei

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are sending a secret letter to a friend who lives in a giant, high-tech library. You don't want the librarian to read your letter, so you decide to write it in a secret code. Maybe you swap every word with a nonsense word, or you mix your sentences with random gibberish, or you add a layer of static noise to the page. This is the basic idea behind "obfuscation" in the world of artificial intelligence. Today, many people use cloud-based AI services to write stories, analyze medical records, or solve complex problems. But since they don't own the computers running these AI brains, they have to trust the company with their private data. To protect themselves, users try to "scramble" their messages before sending them, hoping the AI can still understand the scrambled version to do its job, while keeping the original meaning hidden from prying eyes.

For a while, these scrambling tricks seemed like a solid shield. Researchers proposed various ways to mix up words or add noise, believing that once the message was scrambled, it would be impossible to unscramble it without the secret key. It was like putting a message in a locked box and throwing away the key, assuming the box was too strong to break. But what if the box wasn't as strong as everyone thought? What if there was a clever way to peek inside, not by picking the lock, but by noticing the faint patterns the scrambling left behind? This is the question a team of researchers set out to answer, exploring whether these "scrambled" messages are actually safe or if they are just waiting to be decoded.

Enter DeepInvert, a new digital detective tool created by researchers at Ant Group. Think of DeepInvert as a master puzzle solver who doesn't need the original key to figure out what's inside the scrambled box. Instead of just guessing, it uses a clever two-step strategy. First, it practices on a pile of "shadow" puzzles—messages it knows the answers to, which it scrambles itself to learn how the scrambling works. But here's the magic trick: it also looks at the real, scrambled messages it's trying to crack, even though it doesn't know what they say. It uses a technique called "semi-supervised learning," which is like a student who studies a textbook (the shadow data) but also learns by observing the patterns in the real world (the unlabeled data) to fill in the gaps.

The researchers found that DeepInvert is incredibly good at its job. When they tested it against some of the most popular scrambling defenses, the results were startling. For example, against a top-tier defense called ObfusLM, the best previous attempts could only guess the original words correctly about 26.2% of the time. DeepInvert, however, managed to recover the correct words 73.5% of the time. In a medical question-and-answer test using a massive AI model, the recovery rate jumped to 80.4%, and for an even larger model, it hit 85.2%. The paper suggests that these scrambling defenses are far less secure than people believed. The researchers discovered a frustrating trade-off: if the scrambling is weak enough to let the AI do its job well, DeepInvert can easily unscramble it. If the scrambling is strong enough to stop DeepInvert, the AI often becomes so confused that it can't do the task at all.

The paper doesn't just show that these defenses can be broken; it explains why they fail. The scrambling methods often leave behind a "semantic skeleton"—a hidden structure of meaning that survives the noise. DeepInvert is designed to find that skeleton. It uses a "teacher-student" system where a stable "teacher" model guides a "student" model, helping it learn from the messy, scrambled data without getting confused. The researchers also showed that this attack works even when the attacker doesn't have the exact same "scrambling recipe" as the victim, as long as they can simulate a similar one.

However, the paper is careful not to say that all privacy is gone. It notes that for very simple tasks, like deciding if a sentence is happy or sad, some defenses might still hold up a bit better. But for complex tasks that require understanding specific words, like medical diagnoses or generating new text, the current scrambling methods seem to offer a false sense of security. The authors conclude that we might need to rethink how we protect data in the cloud. Instead of relying on these lightweight scrambling tricks, we may need to look toward heavier, more complex solutions like advanced cryptography, which are harder to break but also much slower and more expensive to use. For now, DeepInvert serves as a loud wake-up call: if you are scrambling your data to hide it from an AI, you might want to double-check your lock, because someone just found a very effective way to pick it.

Technical Summary: DeepInvert

Problem Statement

Cloud-based language model (LM) services frequently process prompts containing sensitive personal or business information. To mitigate privacy risks without the heavy computational overhead of cryptographic solutions (e.g., homomorphic encryption), recent research has proposed obfuscation-based defenses. These methods transform prompt representations on the client side before transmission to the server using techniques such as vocabulary permutation, token substitution, clustering/mixing, and differential privacy (DP) noise addition.

While these defenses aim to preserve downstream task utility while hiding original tokens, the authors argue that existing protections are significantly weaker than believed. Prior attacks, such as KNN-based similarity searches or fully supervised inversion models (e.g., InvBert), have shown limited success. Specifically, supervised attacks suffer from distribution shifts between the attacker's shadow data and the target's obfuscated embeddings, while optimization-based attacks lack structural robustness against obfuscation noise.

Methodology: DeepInvert

The paper presents DeepInvert, a semi-supervised embedding inversion attack designed to recover original tokens from obfuscated representations with higher accuracy than prior methods. The core insight is that while obfuscated embeddings are perturbed, they retain exploitable semantic structure that can be learned from unlabeled target data.

Core Architecture

DeepInvert employs a semi-supervised learning framework inspired by FixMatch but tailored for the high-vocabulary, noisy nature of embedding inversion. It alternates between two training objectives:

Supervised Learning (Shadow Data): The model is trained on labeled obfuscated embeddings generated by "shadow models" (simulated defenses). This component uses standard cross-entropy loss to map obfuscated states to token IDs.
Unsupervised Consistency (Target Data): The model leverages unlabeled obfuscated embeddings from the actual target deployment. It enforces consistency between "weakly" and "strongly" augmented views of the same embedding.

Key Technical Innovations

To address the specific challenges of embedding inversion (large vocabulary, distribution shift, and noise), DeepInvert introduces several tailored adaptations:

Soft Labeling & Top- $k$ KL Divergence: Instead of using hard pseudo-labels (which are often inaccurate in high-vocabulary settings), the attack uses soft labels. It constrains the consistency loss to the top- $k$ predicted probabilities, reducing the influence of low-confidence long-tail predictions and optimizing computational resources.
Masked Augmented Views: Borrowing from pretraining techniques, the method randomly masks token embeddings in the input sequence. This forces the model to reconstruct semantic structure from context rather than relying on specific token features, helping to restore information damaged by obfuscation.
EMA Teacher: To stabilize the unsupervised training and prevent model collapse, an Exponential Moving Average (EMA) teacher model is used. The student model learns from the teacher's predictions on weak views, while the teacher is updated via EMA of the student's parameters.
Mixed Training Pipeline: Rather than jointly optimizing supervised and unsupervised losses (which can lead to gradient conflicts), DeepInvert uses an alternating schedule. It switches between supervised and unsupervised optimization steps within an epoch, allowing the two objectives to mutually reinforce each other.
Defense-Specific Optimizations:
- Multi-Shadow Attack: To mitigate distribution shifts caused by secret permutations or clustering in defenses like ObfusLM, the attack utilizes multiple shadow models with different random seeds.
- Renormalization & Denoising: For DP-based defenses, the attack applies layer normalization to standardize embedding scales and uses Principal Component Analysis (PCA) to denoise embeddings before inversion, effectively reducing the noise-to-signal ratio.

Experimental Results

The authors evaluated DeepInvert across nine defenses, five tasks (including binary/multi-class classification, NER, and medical QA), and four model architectures (BERT, RoBERTa, LLaMA3-8B, Qwen3.5-27B).

Performance on ObfusLM: Against the state-of-the-art ObfusLM defense, DeepInvert achieves a 73.5% top-1 token recovery rate (on SST-2 with RoBERTa), a significant improvement over the previous best attack (InvBert) which achieved only 26.2%. In generation tasks (Medical QA with Qwen3.5-27B), the recovery rate reaches 85.2%.
Generalization: DeepInvert consistently outperforms baselines (KNN, TBS, DEML, InvBert) across most defenses and tasks. It demonstrates robustness even in out-of-domain settings where the shadow data distribution differs from the target data.
Sensitive Token Recovery: The attack recovers sensitive tokens (e.g., medical terms, named entities) at rates comparable to non-sensitive tokens, indicating that obfuscation does not selectively protect sensitive content.

Significance and Claims

The paper claims that DeepInvert exposes a fundamental task-dependent tension in the privacy-utility trade-off of current obfuscation defenses:

Token-Level and Generation Tasks: There is a severe conflict. Defenses that preserve enough signal for downstream utility (e.g., accurate generation or NER) inherently retain sufficient semantic structure for DeepInvert to recover original tokens accurately. Conversely, defenses strong enough to resist inversion (e.g., high $\epsilon$ DP or heavy noise) collapse downstream task utility to near-random levels.
Sentence-Level Classification: On simpler tasks like sentiment analysis, some DP-based defenses can simultaneously maintain utility and limit token recovery, suggesting the tension is less acute for coarse-grained tasks.

The authors conclude that current obfuscation-based defenses provide far less protection than previously believed under standard privacy parameters. They call for a re-evaluation of this class of defenses, particularly for token-level and generative workloads, and suggest that achieving meaningful privacy may require a shift toward heavyweight cryptographic solutions or hybrid designs.