Technical Summary: FuseFSS

Problem Statement

In two-server secure inference for Large Language Models (LLMs), clients can query hosted models without revealing prompts or embeddings by secret-sharing inputs between non-colluding servers. While recent GPU-based systems utilizing Function Secret Sharing (FSS) have made linear layers efficient, significant bottlenecks remain in fixed-point nonlinearities (e.g., activations like GELU, Softmax components) and rescaling operations.

Current state-of-the-art systems (e.g., Sigma) rely on bespoke, per-operator protocols. Each operator requires a custom implementation involving specific comparisons, wrap-around corrections, and preprocessing material. This approach is brittle: it complicates correctness proofs, duplicates key-generation logic, and makes adding new operators difficult without introducing subtle bugs related to modulo-$2^n$ wrap-around or signedness. Furthermore, these custom protocols often lead to high online communication overhead and heavy preprocessing costs (key generation time and size).

Methodology: FuseFSS

The authors present FuseFSS, a compiler that replaces per-operator protocol engineering with a uniform, GPU-friendly compilation pipeline. The core insight is that across common fixed-point nonlinearities, the data-dependent logic follows a similar pattern: selecting a region using a small set of predicate bits and then applying a low-degree arithmetic form with a small number of constants.

1. Operator Specifications

FuseFSS introduces a typed operator specification that serves as an intermediate representation (IR) for elementwise fixed-point operators. A specification defines:

• Interval Partition: A full partition of the canonical representatives of $\mathbb{Z}_{2^n}$ into intervals $I_i = [\alpha_i, \alpha_{i+1})$.
• Arithmetic Pieces: For each interval, a vector of low-degree polynomials $P_i(x)$ to compute the arithmetic output.
• Predicate Bits: Boolean formulas $B_i(x)$ built from primitive predicates (e.g., $x < \beta$) to determine which interval is active.

2. Mask-Aware Compilation

The compiler translates these specifications into a uniform protocol that operates on a public masked value $\hat{x} = x + r_{in} \pmod{2^n}$, where $r_{in}$ is a fresh, secret-shared mask sampled during preprocessing.

• Predicate Rewriting: The compiler rewrites predicates on the secret input $x$ into comparisons on the public $\hat{x}$, utilizing secret-shared carry bits derived from the mask $r_{in}$ to handle modulo wrap-around correctly.
• Uniform Structure: Regardless of the specific operator, the compiler emits exactly two standard FSS evaluations on the public $\hat{x}$:
  1. Packed Comparison: A single evaluation returning XOR shares of all required predicate bits (interval indicators).
  2. Vector Interval Lookup: A single evaluation returning additive shares of the active polynomial coefficients and constants for the selected interval.

3. Post-Processing

The remaining computation is a fixed, share-based post-processing circuit. This includes:

• Horner's Rule: Evaluating the selected low-degree polynomial on the secret shares of $x$ (derived locally from $\hat{x}$ and $r_{in}$) using Beaver triples.
• Helper Operations: Applying Boolean logic (XOR, AND) and mixed-domain conversions (B2A/A2B) to combine arithmetic and boolean results.
• Specification-Compatible Gates: For operations that cannot be purely polynomial (e.g., truncation, arithmetic right shifts), the compiler adds a deterministic post-processing circuit $\Phi$ that takes the polynomial output and mask-derived constants to produce the final result.

4. Security and Leakage

FuseFSS adheres to masked-wire semantics, ensuring fresh independent masks per wire. Crucially, the compiler enforces mask-independent public shapes. The number of comparisons, bit-widths, and interval lookup dimensions depend only on the public operator specification and fixed-point metadata, not on the sampled masks. This prevents leakage of secret information through key sizes or instance shapes. The security model assumes a semi-honest adversary in a two-server preprocessing setting.

Key Contributions

1. State-of-the-Art Performance: FuseFSS achieves the best reported end-to-end performance for FSS-based secure inference on BERT and GPT-style models, improving latency and reducing preprocessing costs while maintaining model accuracy.
2. Practical Compiler Target: It demonstrates that a wide range of fixed-point scalar operators can be unified under a single specification format, executed via two standard FSS calls plus uniform post-processing, eliminating the need for bespoke protocol design.
3. Mask-Aware Compilation with Security: The system derives correct predicate evaluations from public masked inputs while enforcing shape invariance, enabling a unified correctness and semi-honest security proof across different operators.

Experimental Results

The authors evaluated FuseFSS against Sigma, the current state-of-the-art FSS-based baseline, on BERT and GPT-style models (including BERT-tiny/base/large, GPT-2, and GPT-Neo) with sequence length 128 and batch size 1.

• End-to-End Speedup: FuseFSS achieves a 1.24×–1.50× speedup in online latency. For example, BERT-base-128 inference time dropped from 1613.80 ms (Sigma) to 1149.50 ms (FuseFSS), a 28.8% reduction.
• Communication Reduction: Online communication is reduced by 9%–16%.
• Preprocessing Efficiency:
  • Key Generation Time: Reduced by 14%–23%.
  • Key Size: Reduced by 20%–24%.
• Scalability: The speedup remains consistent (1.36×–1.45×) as sequence length increases from 32 to 512.
• Attribution: The gains are primarily driven by the acceleration of scalar nonlinear and helper blocks (e.g., GELU, Softmax components), which account for the majority of the latency in the baseline. For instance, the compiled path for Softmax (nExp + reciprocal) saw a 4.5×–4.7× speedup.

Significance and Claims

The paper claims that FuseFSS addresses the primary bottleneck in two-server secure inference: the engineering complexity and performance overhead of fixed-point nonlinearities. By shifting from bespoke per-operator protocols to a uniform compilation pipeline, FuseFSS simplifies the system design, improves correctness guarantees, and significantly reduces both online and offline costs.

The authors position FuseFSS as a practical solution for deploying privacy-preserving LLM inference in settings requiring strong confidentiality for prompts and embeddings (e.g., healthcare, finance). They explicitly note that their work focuses on scalar nonlinearities; vector-level operations (like reductions or attention sparsification) remain outside the operator specification IR and are handled by standard MPC subprotocols. The security analysis is confined to the semi-honest model with two non-colluding servers, acknowledging that extending this to stronger adversarial models is a future direction.