🤖 AI

WHALE: A Scalable Unified Model for Recommendation with Wukong-HSTU Architecture

The paper introduces WHALE, a scalable unified recommendation architecture that integrates Wukong and HSTU backbones through a novel fusion mechanism to jointly model non-sequence and sequence features, achieving significant offline and online performance gains in industrial production systems.

Original authors: Renqin Cai, Dawei Sun, Yuanjun Yao, Zhiyong Wang, Velvin Fu, Maggie Zhuang, Yu Shi, Zhongnan Fang, Xuan Cao, Jing Qian, Rui Li

Published 2026-07-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Renqin Cai, Dawei Sun, Yuanjun Yao, Zhiyong Wang, Velvin Fu, Maggie Zhuang, Yu Shi, Zhongnan Fang, Xuan Cao, Jing Qian, Rui Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are walking through a massive, bustling digital city where millions of people are constantly scrolling, clicking, and watching. This is the world of online recommendation systems, the invisible engines behind your social media feeds, video suggestions, and shopping lists. Their job is to guess what you want to see next. To do this, they usually look at two very different types of clues. First, there are your "static" details: who you are, what you like, the time of day, and the specific item being shown. It's like a detective looking at a suspect's file and the crime scene. Second, there are your "dynamic" actions: the long, messy trail of everything you've clicked, watched, or bought in the past. This is like a detective watching a suspect's entire life story unfold in real-time.

For a long time, recommendation systems treated these two clues separately. Some super-smart models were great at mixing the static details (like "this user likes sci-fi" + "it's Friday night"), while other models were experts at reading the long story of your past behavior (like "you watched three space movies in a row, so you probably want a fourth"). The big question was: could we build one super-model that was equally brilliant at both, without getting too slow or too expensive to run? If we could, it might mean seeing content that feels perfectly tailored to exactly who you are and what you've been doing lately, all at the same time.

Enter WHALE, a new architecture developed by researchers at Meta that tries to solve this puzzle. Think of WHALE as a giant, multi-layered team of detectives working together. Instead of having one detective read the file and another watch the video, WHALE puts them in the same room, layer by layer. In every single layer of the model, there is a "Wukong" team (the expert at mixing static details) and an "HSTU" team (the expert at reading the long history of your actions). But here is the magic trick: they don't just work side-by-side; they constantly chat. The Wukong team asks the HSTU team, "Hey, based on this specific item I'm looking at, which part of the user's long history is most important right now?" The HSTU team then zooms in on that specific memory and hands it back. This happens over and over again as the data moves through the model, allowing the system to constantly refine its understanding.

The paper suggests that this "progressive exchange" is a game-changer. When the researchers tested WHALE on massive amounts of real-world data from a social media platform, they found that it consistently got better at predicting what users would click on as they made the model bigger and gave it longer history to read. Specifically, when they increased the length of the user history the model could see, the quality of recommendations improved. They also found that making the model deeper (more layers) and wider (more processing power) continued to yield better results, suggesting the system scales up well.

However, the researchers also tested a simpler way to combine these two types of models, which they call a "shallow-hybrid" approach. In this older method, the history is compressed into a tiny summary before being mixed with the static details. The paper argues that this is a mistake because it throws away the fine-grained details. Their experiments showed that WHALE's deep, layer-by-layer chatting was significantly better than this shallow approach, proving that keeping the two teams in constant communication is crucial.

Of course, building such a complex system is expensive. The team had to invent special, custom computer code (using something called "Triton kernels") to make sure WHALE could run fast enough to be used by millions of people at once. In real-world tests, WHALE did improve the main metrics for user engagement, but it did come with a small cost: the system could handle about 5% fewer requests per second compared to the old system. Despite this slight slowdown, the improvement in recommendation quality was considered a win, and the system has already been deployed in production. The paper concludes that while we can't just make models infinitely big to solve everything, unifying these two different ways of thinking about data—static details and dynamic history—into a single, interacting architecture is a powerful and practical path forward for the future of recommendations.

Technical Summary: WHALE - A Scalable Unified Model for Recommendation with Wukong-HSTU Architecture

1. Problem Statement

Large-scale recommender systems must model two distinct but complementary sources of ranking signals: non-sequence features (user/item attributes, context, and explicit crosses) and sequence features (user behavior histories).

Current Limitations: Existing architectures typically treat these signals separately. Wukong architectures excel at scaling high-order interactions among non-sequential features but lack mechanisms to model temporal evolution in behavior sequences. Conversely, HSTU (Hierarchical Sequence Transduction Unit) architectures effectively model long, heterogeneous behavior sequences but do not explicitly scale high-order static and contextual crosses among user, item, and request features.
The Gap: Shallow-hybrid approaches, which compress behavior histories into fixed summaries before feeding them into feature-interaction models, create a bottleneck. Once compressed, high-order feature interactions can no longer selectively retrieve fine-grained evidence from specific events in the user's history. This limits the "progressive exchange" between static feature interactions and dynamic sequential patterns, which is critical for relevance in scenarios where different candidates require different parts of the same history.

2. Methodology: The WHALE Architecture

The authors propose WHALE (Wukong-HSTU-based scALable unified modEl), a recursive architecture that unifies Wukong and HSTU within a single model, keeping both backbones active throughout the network.

2.1 Core Architecture

WHALE consists of an input layer followed by a stack of recursive WHALE layers. Each layer contains three distinct modules:

Wukong Module: Operates on the non-sequence branch (user, item, context, and cross features). It utilizes stacked factorization-machine blocks (FMB) and linear compression blocks (LCB) to construct high-order interaction representations ( $H_w$ ).
HSTU Module: Operates on the sequence branch (user behavior history). It uses attention-based mechanisms with relative attention bias to encode fine-grained temporal evidence and dependencies, producing sequence-aware behavior representations ( $H_s$ ) that retain the full sequence length ( $L$ ).
Attention-Based Fusion Module: This is the core innovation enabling progressive exchange.
- Mechanism: It treats the Wukong interaction representations ( $H_w$ ) as queries and the HSTU behavior representations ( $H_s$ ) as keys and values.
- Function: This allows candidate- and context-dependent feature crosses to selectively retrieve the most relevant behavioral evidence from the user's history.
- Refinement: The attention output is concatenated with the original Wukong representation, projected via a Fusion MLP, and passed through a pre-norm SwiGLU FFN (gated feed-forward network) to produce a fused representation ( $H_f$ ).
- Propagation: The fused representation ( $H_f$ ) becomes the non-sequence input for the next layer, while the HSTU output ( $H_s$ ) propagates independently to the next layer's sequence module. This design ensures that static interactions and dynamic patterns condition on each other repeatedly across layers.

2.2 Efficiency Optimizations

To make WHALE viable for industrial deployment, the authors implemented several system-level optimizations:

Customized Triton Kernels:
- Shared Key-Value: Since cross-branch attention is memory-bound, the authors share the Key and Value tensors ( $K=V$ ), halving KV-side GPU memory traffic.
- Asymmetric Backward Variants: Recognizing the asymmetric shape (few queries vs. long history), the system dynamically selects between a KV-parallel or Q-parallel backward pass schedule to minimize gradient reduction costs.
Shared-Gate SwiGLU FFN: The gate and up-projection matrices in the SwiGLU FFN are tied ( $W_{gate} = W_{up}$ ), reducing matrix multiplications from three to two without degrading quality.
Mixed-Precision & Compiler Optimization: The dense backbone runs in BF16, while sensitive components (loss, task arch) remain in FP32. torch.compile is used to fuse operators and optimize memory usage.
Inference Optimizations: Includes AOTInductor for operator fusion and "shape hint tensors" to eliminate GPU-CPU synchronization bottlenecks during dynamic shape calculations, reducing latency by 1–5 ms.

3. Key Contributions

Unified Architecture: WHALE is presented as the first scalable recommendation architecture to unify Wukong (non-sequential feature interaction) and HSTU (sequential behavior modeling) within a single model, utilizing progressive layer-wise cross-branch exchange rather than shallow coupling.
System-Model Co-Design: The development of customized kernels and training/inference optimizations that enable the online deployment of such a complex, unified model under strict latency and throughput constraints.
Empirical Validation: Demonstration that WHALE scales effectively with sequence length, model depth, and model width, outperforming strong single-paradigm baselines in both offline and online settings.

4. Experimental Results

Experiments were conducted on a large-scale short-form video social media platform (80B training samples, 4B evaluation samples).

Offline Performance (Scaling):
- Architecture Comparison: WHALE consistently outperformed Wukong-only and HSTU-only baselines across varying model complexities (8, 14, and 32 GFLOPs).
- Scaling Laws: WHALE showed consistent gains as sequence length increased (up to 15k), model depth increased (up to 8 layers), and model width increased (up to 512 dimensions).
- Ablation Studies: Removing the attention mechanism (replacing with average pooling) or removing the Fusion MLP/SwiGLU FFN resulted in significant quality drops, confirming the necessity of the specific fusion design.
Online Performance (A/B Test):
- Deployed in production for 14 days.
- Primary Metric: Achieved a +0.113% gain over the online baseline.
- Secondary Metrics: Gains of +0.824% and +1.820% on supporting metrics.
- Trade-off: The model incurred a -5% regression in inference QPS (throughput), which the authors note remained acceptable within the serving budget.

5. Significance and Claims

The paper claims that WHALE provides a practical example of how two distinct sources of information—static feature interactions and dynamic sequential behaviors—can be scalably unified in an industrial setting.

The authors argue that the next stage of scaling industrial recommenders may not come solely from making individual backbones larger, but from progressively unifying complementary architectures at each layer. By allowing high-order feature crosses to repeatedly retrieve fine-grained evidence from long user histories, WHALE addresses the limitations of shallow hybridization and demonstrates that deep, cross-branch interaction is a viable and effective design principle for large-scale ranking systems.