T-REX: Transformer-Based Category Sequence Generation for Grocery Basket Recommendation

The paper proposes T-REX, a novel transformer-based architecture that addresses the unique challenges of online grocery shopping by generating personalized category-level basket recommendations through dynamic sequence splitting, adaptive positional encoding, and causal masking to effectively capture both short-term dependencies and long-term user preferences.

The Gist

Imagine you are walking through a massive, infinite grocery store. Every time you shop, you have a mental list of things you need: milk, eggs, bread, maybe some pasta. But here's the catch: you don't just buy random things. You have habits. You always buy milk before you buy cereal. You tend to buy fresh vegetables on Tuesdays.

For a long time, online grocery stores tried to guess what you wanted by looking at your past receipts and saying, "Hey, you bought milk 10 times, so here's milk again!" This is called the "Personal Top Frequency" (P-Top) method. It's like a very boring, repetitive shop assistant who only remembers your most frequent purchases but doesn't understand the story of your shopping trip.

The paper you shared introduces T-REX, a new, super-smart AI assistant designed specifically for this chaotic, repetitive world of grocery shopping. Here is how it works, explained simply:

1. The Problem: Why Old Methods Fail

Traditional online shopping (like buying a TV or a sofa) is about one-off purchases. You buy a TV, and you're done. Grocery shopping is different; it's a loop. You buy the same things over and over, but in different combinations.

Old AI models (like the ones used for reading text or translating languages) tried to look at your whole history at once, like reading a book from start to finish. But in grocery shopping, that's a problem. If the AI sees you bought "milk" at the end of your history, it might accidentally "peek" at that future fact while trying to guess what you need now. It's like a student cheating on a test by looking at the answer key before writing the question.

2. The Solution: Meet T-REX

T-REX is a Transformer model (the same type of brain behind advanced chatbots), but it's been customized for groceries. Think of it not as a student taking a test, but as a personal shopper who walks with you down the aisle.

Here are its three superpowers:

A. The "Dynamic Split" (The Time-Traveling Pivot)

Instead of looking at your whole history as one big block, T-REX picks a random moment in your past (a "pivot point").

• The Analogy: Imagine you are watching a movie of your life. T-REX pauses the movie at a random scene. It looks at everything that happened before the pause (the Encoder) to understand your habits. Then, it tries to predict what happens immediately after the pause (the Decoder).
• Why it helps: This teaches the AI to predict the next step based on what came before, without cheating by looking at the future. It also helps it learn from new customers who don't have a long history yet, because it can learn from just a few scenes of their "movie."

B. The "Adaptive Clock" (Understanding Time)

In a text message, every word follows the one before it instantly. In grocery shopping, you might buy milk on Monday, then buy bread three weeks later.

• The Analogy: Standard AI clocks tick every second. T-REX has a flexible, rubber-band clock. It understands that "3 days ago" and "3 weeks ago" are both part of your history, but they feel different. It adjusts its understanding of time so it doesn't get confused by the gaps between your shopping trips.

C. The "Category Map" (Thinking in Groups)

There are 29,000 different products in a grocery store. Trying to guess the exact next brand of cereal is hard.

• The Analogy: Instead of guessing "Kellogg's Corn Flakes," T-REX thinks in categories like "Breakfast Cereal" or "Dairy."
• Why it helps: It's like organizing a messy closet. Instead of trying to remember the exact location of every single sock, you just know "socks go in the drawer." This makes the AI faster, smarter, and less likely to get overwhelmed by the sheer number of products.

3. The Result: A Better Shopping Experience

The researchers tested T-REX against the old "boring shop assistant" (P-Top) in real Amazon grocery stores.

• The Score: T-REX didn't just guess the right items; it guessed the right order.
• The "Rank-Matching" Magic: Imagine your shopping list has a hierarchy. You need milk (Rank 1), then eggs (Rank 2). The old system might suggest eggs first because you buy them often. T-REX understands that you need milk first. It aligns its suggestions with how you actually build your basket.
• The Business Win: Because the suggestions were better, people bought more. The system increased sales by about 23% more than typical recommendation systems.

The Big Picture

Think of T-REX as the difference between a robot that reads a spreadsheet and a human friend who knows your routine.

• The Robot says: "You bought milk 50 times. Here is milk."
• The Friend (T-REX) says: "You usually buy milk on Tuesdays, and since you just bought bread, you probably need butter next. Also, it's getting cold, so maybe some soup?"

By understanding the sequence of your life and the categories of your needs, T-REX makes online grocery shopping feel less like a chore and more like a helpful conversation.

Technical Summary

Here is a detailed technical summary of the paper "T-REX: Transformer-Based Category Sequence Generation for Grocery Basket Recommendation."

1. Problem Statement

The paper addresses the challenge of Next Basket Recommendation (NBR) in the online grocery domain. Unlike traditional e-commerce (which often focuses on one-off, exploratory purchases), grocery shopping is characterized by:

• Repetitive Patterns: Customers buy similar items regularly.
• Complex Relationships: Items within a basket are complementary (e.g., milk and cereal) rather than just sequentially related.
• Temporal Irregularity: Shopping sessions are unordered sets of items that occur at irregular intervals (days or weeks apart).

Key Challenges:

• Data Sparsity: Direct product-level modeling suffers from high dimensionality (29,000+ products) and sparse purchase histories.
• Baseline Limitations: Simple frequency-based baselines (like Personal Top-Frequency or P-Top) often outperform complex neural models in grocery settings because customer habits are stable.
• Model Suitability: Standard bidirectional transformers (like BERT4Rec) suffer from information leakage in basket generation. They can "peek" at future occurrences of the same item during training, leading to unrealistic performance that fails to generalize to real-world sequential prediction.

2. Methodology: T-REX Architecture

The authors propose T-REX, a custom Transformer architecture designed specifically for generating personalized category-level shopping baskets.

A. Problem Formalization & Data Transformation

• Category-Level Modeling: Instead of predicting specific SKUs, the model maps 29,000+ products to 35 (later 280) high-level categories (e.g., "Dairy," "Produce"). This reduces dimensionality, stabilizes temporal patterns, and aligns with how customers mentally organize shopping lists.
• Sequence Definition: A customer's history is a temporally ordered sequence of baskets $S(u) = (S_1, ..., S_t)$. The goal is to predict the probability distribution of the next basket $S_{t+1}$ given the history.

B. Architectural Innovations

T-REX utilizes an Encoder-Decoder structure (unlike the single-module BERT4Rec) for three specific reasons:

1. Encoder: Learns generalizable basket composition patterns across the entire customer base. It processes the full history without masking to capture global dependencies.
2. Decoder: Generates personalized predictions using causal masking (only attending to past items). It leverages cross-attention to connect the encoder's global patterns with the individual's specific history.
3. Causal Masking: Unlike BERT's bidirectional masking, T-REX uses causal masking to prevent information leakage, ensuring the model learns true sequential dependencies suitable for next-basket generation.

C. Key Technical Components

1. Dynamic Sequence Splitting:

   • Instead of fixed session splitting, the training data is split dynamically based on a random pivot timestamp ($t_p$).
   • Purchases before $t_p$ form the Encoder Input; purchases after $t_p$ form the Decoder Target.
   • Benefit: Creates diverse training contexts, handles varying history lengths, and enables mid-session recommendations.

2. Adaptive Positional Encoding:

   • Standard sinusoidal encodings fail because grocery sessions are irregular (days/weeks apart).
   • T-REX uses Normalized Relative Positions: The position of an item is encoded as the number of days before or after the pivot purchase. This captures temporal density without being overwhelmed by long gaps.

3. Training Strategy:

   • Teacher Forcing: The decoder is trained to predict the next category based on previous categories and the encoder context.
   • Loss Function: Cross-entropy loss over the category vocabulary.
   • Hyperparameters: 6-layer encoder/decoder, 6 attention heads, hidden dimension 2048, embedding dimension 512 (~50.4M parameters).

3. Key Contributions

1. Novel Architecture: Introduction of an Encoder-Decoder Transformer specifically tailored for grocery basket generation, solving the information leakage issues found in bidirectional models like BERT4Rec.
2. Dynamic Sequence Splitting: A novel sampling strategy that improves generalization for new customers and handles sparse, irregular shopping patterns better than session-based splitting.
3. Adaptive Positional Encoding: A time-aware encoding scheme that effectively models the irregular temporal intervals inherent in grocery shopping.
4. Rank-Matching Evaluation: A new metric that evaluates not just if an item is recommended, but how well the ranking of recommendations aligns with the customer's historical purchase frequency (e.g., ensuring the most frequent category is ranked #1).

4. Experimental Results

The model was evaluated on a dataset of 1 million shopping sessions from 100,000 customers.

• Baselines: Compared against P-Top (Personal Top-Frequency), a strong baseline that often outperforms complex models in grocery domains.
• Offline Metrics:
  • T-REX outperformed P-Top across all $k$ values for Precision@k and Recall@k.
  • Consistency: T-REX showed lower variance in performance (tighter box plots), indicating it learns generalizable patterns rather than just memorizing frequencies.
  • Rank-Matching: T-REX achieved 30% exact rank matches (vs. 27% for P-Top) and 60% within one rank (vs. 48% for P-Top), proving it better predicts the order of importance for categories.
• Segment Analysis:
  • New Customers: T-REX showed significant gains for customers with short histories, leveraging population-level patterns learned by the encoder.
  • Small Baskets: T-REX maintained a 10–30% advantage for very small baskets (<5 items), where frequency baselines struggle due to lack of data.

5. Production Deployment & Significance

• A/B Testing: T-REX was deployed in Amazon's grocery business for Consumables recommendations.
• Business Impact:
  • Achieved the highest Click-Through Rate (CTR) among comparable widgets.
  • Delivered a 23% higher sales lift compared to typical Amazon-wide recommendation systems.
  • Improved order product sales, composite sales, and gross contribution profit.
• Significance:
  • The paper demonstrates that while simple frequency models are strong baselines in grocery, sequential deep learning can surpass them if the architecture is correctly aligned with the domain's unique constraints (irregular timing, category-level planning, and causal generation).
  • It validates the shift from product-level to category-level modeling for scalability and user intent alignment in large-scale grocery systems.