Technical Summary: Privacy from Symmetry (CONJFORMER)

Problem Statement

The deployment of Large Language Models (LLMs) as cloud-based services necessitates sending user data to third-party providers, creating significant privacy risks. While Split Inference (SI) mitigates this by keeping the embedding layer local and transmitting only intermediate hidden representations, it remains vulnerable to nearest-neighbor (NN) embedding inversion. In standard SI settings, the embedding table is public, allowing an adversary to recover input tokens by matching server-observed hidden states against the public table using cosine similarity. This attack remains highly effective even after model fine-tuning. Existing privacy-preserving methods, such as differential privacy (DP) noise injection or cryptographic techniques (Homomorphic Encryption, Secure Multiparty Computation), suffer from either significant model quality degradation or prohibitive computational overhead, making them impractical for autoregressive LLM inference.

Methodology: CONJFORMER

The paper proposes CONJFORMER, a framework that enforces $O(d)$-equivariance (equivariance to the orthogonal group of dimension $d$) at the architectural level. This allows the client to obfuscate embeddings using a secret orthogonal matrix while ensuring the server can still perform correct inference without ever observing unrotated hidden states.

Core Mechanisms

1. Orthogonal Obfuscation: The client multiplies input embeddings by a secret orthogonal matrix $U \in O(d)$ before transmission.
2. Weight Conjugation: To preserve model functionality under this rotation, the client applies two-sided linear conjugation to all linear weights in the transformer. If a linear layer is $W$, the server receives $\hat{W} = O W U^\top$, where $O$ is a block-specific orthogonal matrix.
3. Architectural Modifications: To achieve exact $O(d)$-equivariance, the authors introduce minimal changes to the transformer architecture:
   • Scalar RMSNorm: Standard normalization layers (LayerNorm or RMSNorm with learnable vectors) are coordinate-dependent and break equivariance. CONJFORMER replaces these with RMSNorm using a scalar gain. This ensures the normalization operation commutes with orthogonal transformations.
   • Blockwise Conjugation: All linear projections (Query, Key, Value, Output, and FFN weights) are conjugated. The server performs the entire forward pass in the rotated basis.

Theoretical Guarantee

The framework ensures blockwise functional equivalence:
$$\hat{F}_b(X U^\top) = F_b(X) U^\top$$
where $F_b$ is the original transformer block and $\hat{F}_b$ is the conjugated block. Consequently, the server computes the exact same logic as the original model but operates entirely on rotated representations. The server never observes the unrotated hidden states, rendering direct NN inversion against the public embedding table impossible.

Key Contributions

1. Architecture: Development of a transformer variant that is exactly $O(d)$-equivariant via lightweight modifications (scalar RMSNorm) and weight conjugation, compatible with arbitrary orthogonal transformations of the embedding space.
2. Framework: Proposal of the CONJFORMER framework, which applies blockwise orthogonal conjugation to preserve functionality under secret obfuscation, effectively removing direct nearest-neighbor embedding inversion.
3. Attack Analysis: Identification and analysis of new attack vectors necessitated by the obfuscation. Since direct matching is eliminated, the adversary must resort to weight alignment attacks to recover the hidden orthogonal transformations. The paper introduces and evaluates:
   • Two-sided ALS–Procrustes Alignment: An iterative method to estimate $U$ and $O$ by aligning conjugated fine-tuned weights with public pretrained weights.
   • Gram-based Weight Alignment: A method that aggregates information across blocks by exploiting the invariance of Gram matrices under block-specific rotations, offering a stronger attack than Procrustes.
4. Empirical Validation: Extensive experiments on GPT-2 (Small, Medium, Large) and Llama 3.2 1B models fine-tuned on PubMed.

Experimental Results

The authors evaluated CONJFORMER on GPT-2 variants and Llama 3.2 1B, fine-tuned on PubMed abstracts.

• Privacy Efficacy:
  • Original Models: Nearest-neighbor embedding inversion achieves high accuracy (over 35% top-10 token recovery) even after fine-tuning.
  • CONJFORMER: Orthogonal obfuscation eliminates direct cosine NN inversion. The strongest remaining attacks (Gram-based weight alignment) degrade to near-random performance (at most 1.3% top-10 token recovery) after fine-tuning.
• Model Performance:
  • Perplexity: The framework incurs minimal performance cost. Fine-tuned CONJFORMER models show perplexity increases of only 0.4% (Llama 3.2 1B) to roughly 2% (GPT-2 Small) compared to baseline models.
  • Retrofitting: Pretrained models can be "retrofitted" to the equivariant architecture by replacing normalization layers and performing brief fine-tuning, recovering original perplexity within 2,000 iterations.

Significance and Claims

The paper claims that CONJFORMER provides a practical defense for privacy-preserving LLM inference without relying on noise injection (which degrades quality) or heavy cryptographic machinery (which is too slow).

• Shift in Adversarial Problem: The primary significance is that the method fundamentally changes the nature of the attack. It moves the adversary from a simple "lookup" problem (matching vectors to a public table) to a complex "basis recovery" problem (solving for hidden orthogonal transformations via weight alignment).
• Architectural Symmetry: The work demonstrates that enforcing symmetry at the architectural level can provide robust privacy guarantees against embedding inversion.
• Limitations: The authors explicitly state that CONJFORMER is not a cryptographic or differentially private mechanism. It relies on the secrecy of the client's orthogonal key and the privacy of the fine-tuned embedding table. It does not protect against attacks that exploit orthogonal invariants (e.g., norms, pairwise distances) or known-plaintext scenarios where the adversary can observe the unrotated inputs. The security relies on the computational difficulty of the weight alignment problem rather than information-theoretic secrecy.