🤖 machine learning

Gecko: Fast Private Inference via Secure Public Encoder Offloading

Gecko is a fast and secure private inference framework that offloads a frozen public encoder while using fixed projections and private feature gating to prevent feature-space extraction attacks, achieving low-latency inference with minimal communication overhead.

Original authors: Cheng'an Wei, Kai Chen, Yue Zhao, Congyi Li, Shenchen Zhu

Published 2026-08-04

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Cheng'an Wei, Kai Chen, Yue Zhao, Congyi Li, Shenchen Zhu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you're trying to send a secret message to a friend who runs a super-smart, locked-down factory. You want them to use their fancy machine to solve a problem for you, but you don't want them to see what you're sending, and you don't want them to steal their own secret recipe for how the machine works. This is the world of Private Inference, a branch of computer science dedicated to keeping both your data and the company's AI models safe.

To do this, scientists use a magical tool called Homomorphic Encryption. Think of it like a locked, transparent box. You can put your data inside, and the factory can run its machine on the box without ever opening it. The machine crunches the numbers while they are still locked up, and when it's done, the result comes out of the box still locked. You take it home, unlock it, and see the answer. The problem? This "locked box" math is incredibly slow and heavy. It's like trying to run a marathon while wearing a suit made of lead bricks. For a standard AI model, it can take over a minute just to look at a single picture.

Recently, some engineers tried a shortcut: "Why not let you do the heavy lifting of recognizing the picture before you send it to the factory?" They suggested you run a generic, public AI on your own phone to turn the picture into a simple list of numbers (a "feature"), send just that list to the factory, and let the factory only do the final, tiny step of guessing the answer. This sounds great for speed, but there's a catch. If you send that list of numbers too clearly, a sneaky spy might be able to trick the factory into revealing its secret recipe much easier than before. It's like giving a thief a map that skips the hard parts of the maze and shows them the secret tunnel straight to the treasure.

This is where Gecko comes in. The researchers behind Gecko realized that while we can offload the heavy work to the user to make things fast, we have to be very careful not to hand the spy a "shortcut." They built a new system that keeps the speed but locks the secret tunnel.

The Gecko Solution: A Playful Breakdown

The Gecko team realized that the old "shortcut" method was too blunt. It usually took the final, most specific list of numbers from the public AI and sent it straight to the factory. This was dangerous because that list was so specialized that it made it easy for a thief to reverse-engineer the factory's secret steps.

Gecko changes the game by acting like a super-smart translator that speaks in riddles. Here is how it works, step-by-step:

1. The Multi-Layered Sketch (The Public Encoder)
Instead of sending just the final, polished drawing of the picture, Gecko asks the public AI to send everything it sees at different levels of detail. Imagine looking at a dog. A simple AI might just say "Dog." But Gecko asks for the sketch of the fur, the shape of the ears, the texture of the nose, and the overall pose all at once. This creates a much richer, more complex picture of the input.

2. The Magic Shrink Ray (Fastfood Projections)
Sending all those details would be too much data (like sending a whole library instead of a postcard). So, Gecko uses a clever mathematical trick called "Fastfood projections." Think of this as a magic shrink ray that compresses all those detailed sketches into a compact, scrambled code. It keeps all the important information but makes it impossible to read without the key. Crucially, this shrink ray is fixed and public; it's not a secret, but it's also not a shortcut.

3. The Secret Filter (Hierarchical Feature Gating)
Now, the factory receives this compressed, scrambled code. But wait! The code has different parts with different "loudness" (some details are huge, some are tiny). If the factory just reads them as-is, the loud parts might drown out the quiet, important ones. Gecko adds a tiny, private "volume knob" layer right before the factory's secret brain. This layer, called Hierarchical Feature Gating (HFG), quietly adjusts the volume of each detail so the factory can hear everything clearly. This part is learned and kept secret, ensuring the factory doesn't just rely on the loudest signal.

4. The Final Lock (Private Predictor)
Only this tiny, adjusted, and scrambled code goes into the factory's secret machine (the private predictor). The factory does its work, and the result is sent back. Because the factory only sees the scrambled, multi-layered code, a spy trying to steal the secret recipe has a much harder time. They can't just look at the final drawing; they have to figure out how to unscramble the riddles first.

What Did They Find?

The Gecko team tested this idea on a bunch of different tasks, from recognizing cats and dogs in photos to identifying sounds in audio files and even spotting diseases in medical images.

Speed: They found that Gecko is incredibly fast. Instead of taking minutes, the whole process takes between 0.4 and 2.2 seconds. That's fast enough to use in real life!
Data Size: The amount of data sent back and forth is tiny, at most 10.8 MB.
Accuracy: Despite all the scrambling and shrinking, Gecko is just as good at guessing the right answer as the standard, slower methods. It didn't lose any "smarts" to gain speed.
Security: This is the big one. They simulated sneaky spies trying to steal the factory's secret recipe. They found that even if the spy knows exactly how the public AI works and tries to use the "shortcut" method, they don't get any easier at stealing the secret. In fact, trying to reuse the public parts actually made the spy's job harder or no better than before. The "feature-space shortcut" that other methods created was successfully blocked.

The Verdict

The paper suggests that Gecko is a promising way to make private AI actually usable. It proves that you don't have to choose between speed and security. By carefully designing how we share information—keeping the public part rich but scrambled, and the private part tiny and protected—we can have our cake and eat it too.

However, the authors are careful to note that this isn't a magic wand that solves every problem. They tested it on image and audio tasks using specific types of AI models (like ResNet and MobileNet). They admit that future work will need to see if this works for the newer, more complex AI models (like the ones that write stories or generate art). But for now, Gecko shows us a clear path forward: a fast, private, and secure way to use the cloud without giving away our secrets.

Technical Summary: Gecko: Fast Private Inference via Secure Public Encoder Offloading

1. Problem Statement

Private inference aims to protect both user inputs and server models during neural network inference, typically using Fully Homomorphic Encryption (FHE). However, the computational cost of performing deep learning operations under encryption is prohibitive; even optimized systems require approximately one minute to process a single ResNet50 image.

Recent approaches attempt to mitigate this by offloading a generic, pretrained "public encoder" (e.g., a backbone) to run in plaintext outside the FHE boundary, encrypting only a lightweight, task-specific predictor. While efficient, this design introduces a critical security vulnerability: the feature-space shortcut.

If the public encoder exposes a selective, final-layer representation, an adversary may learn a simpler mapping from these exposed features to the output, rather than learning the full input-to-output behavior of the original model. This "shortcut" reduces the difficulty of model extraction attacks compared to standard black-box access. The challenge is to design a system that retains the efficiency of offloading without creating this incremental extraction risk.

2. Methodology: The Gecko Framework

Gecko is a private-inference framework designed to limit the feature-space shortcut while maintaining a compact encrypted predictor. It operates through three distinct stages:

A. Public Encoding (Client-Side, Plaintext)

Instead of relying solely on the final layer of a frozen backbone, Gecko aggregates hierarchical features from multiple abstraction levels (low, mid, and high-level) of a frozen, task-agnostic pretrained backbone (e.g., ResNet50v2 or MobileNetV3).

Compression: Raw multi-level features are too large for direct transmission. Gecko applies fixed Fastfood projections (random linear projections) to compress each selected feature map into a fixed dimension $D$ .
Information Preservation: The system uses Normalized Preserved Mutual Information (NPMI) as an offline diagnostic to ensure the compressed representation retains sufficient input information to prevent the feature-space shortcut.

B. Hierarchical Feature Gating (Private, FHE)

The compressed features from different layers often have varying magnitudes and statistical properties, which can destabilize training or cause the model to ignore informative but smaller-scale features.

Gecko introduces a Hierarchical Feature Gating (HFG) layer within the private predictor. This layer learns non-negative weights to rescale the contribution of each feature group before they are fused.
Crucially, once training is complete, the HFG weights are absorbed into the first fully connected layer of the predictor. This ensures that the encrypted inference phase requires no additional cryptographic operations for gating, maintaining low FHE depth.

C. Output Mapping (Client-Side, Plaintext)

To prevent the leakage of unnecessary degrees of freedom (such as absolute logit offsets), Gecko applies a bijective centered-logit mapping. The server returns centered logits (subtracting the mean), which the client decrypts and applies a fixed softmax to recover the final class probabilities. This ensures the interface reveals exactly the information needed for the prediction without exposing extra artifacts.

3. Key Contributions

Secure Partitioning: Gecko proposes a partition that executes an independently public encoder outside FHE while keeping the task-specific predictor under FHE. This substantially reduces protected computation without exposing proprietary task adaptation.
Compact Hierarchical Representation: By combining multi-level features, fixed Fastfood projections, and private feature gating, Gecko creates a rich representation that avoids the "selective final-layer interface" prone to shortcuts, all while keeping the encrypted predictor lightweight.
Formal Design Criteria: The authors formalize two ideal conditions for public offloading:
- Prediction-Artifact Independence: The public component must not contain task-specific private artifacts.
- Information Preservation: The exposed interface must not collapse the inference behavior (i.e., it must preserve the uncertainty of the original input-output mapping).
Empirical Validation: The paper evaluates the system across image and audio tasks, demonstrating that reusing the offloaded public encoder provides no significant advantage to model-extraction adversaries compared to standard black-box access.

4. Experimental Results

The evaluation covers diverse tasks including natural images (CIFAR-10, CIFAR-100, Caltech101), biomedical images (various modalities), and audio (FSDD).

Efficiency: Gecko achieves inference times of 0.4–2.2 seconds with communication costs capped at 10.8 MB. This represents a massive reduction compared to end-to-end FHE baselines, which can take hundreds of seconds (e.g., 188s for ResNet50 in hybrid settings).
Accuracy: Gecko maintains accuracy comparable to transfer-learning baselines (often within 1% difference) across all tested tasks and backbone choices (ResNet50v2 and MobileNetV3).
Security (Extraction Risk):
- Baseline Comparison: Attackers achieved lower surrogate accuracy against Gecko targets than against standard transfer-learning baselines.
- Component Reuse: Experiments where attackers reused the public backbone or the exact projection parameters showed no consistent extraction advantage over free surrogate tuning.
- Ablation: Replacing Gecko's multi-level representation with a final-layer representation (a "shortcut") significantly increased attack accuracy (by 19–27 percentage points), validating that Gecko's design successfully mitigates this specific risk.
- Partial Weight Exposure: In contrast, exposing even partial private model weights (an insecure baseline) drastically improved attack accuracy, highlighting the importance of Gecko's strict separation of public and private components.

5. Significance and Claims

The paper claims that Gecko demonstrates the feasibility of public-encoder offloading as a practical route to reducing encrypted computation while retaining model-side protection.

Theoretical Guarantee: By adhering to information-theoretic criteria (independence and preservation), Gecko ensures that exposing the public encoder does not reduce the prediction-functional uncertainty of the proprietary model.
Practicality: The system achieves sub-second to low-second latency on standard hardware, making private inference viable for sensitive applications like biomedical image classification and audio processing.
Robustness: The evaluation suggests that the "feature-space shortcut" is a real and significant risk in offloading designs, but it can be effectively neutralized through hierarchical feature aggregation and careful representation design.

The authors conclude that efficient private inference requires both cryptographic optimization and an explicit, secure separation between independently public resources and proprietary functionality.