Scaling Multilingual Semantic Search in Uber Eats Delivery

This paper presents a production-oriented, unified multilingual semantic retrieval system for Uber Eats that leverages a fine-tuned Qwen2 two-tower model with advanced loss functions and Matryoshka Representation Learning to achieve significant recall improvements across stores, dishes, and grocery items.

The Gist

Imagine you walk into a massive, chaotic library that contains everything from pizza recipes to grocery lists and restaurant menus for the entire world. In the past, if you asked the librarian, "I want something spicy," they might only look for the exact words "spicy" in the book titles. If you asked in Spanish, they might not understand you at all. If you wanted a specific dish and a specific grocery item, they'd have to run to two different sections of the library, slowing everything down.

This paper describes how Uber Eats built a new, super-smart librarian (a search engine) that solves all these problems at once. Here is the story of how they did it, explained simply.

1. The Problem: Too Many Librarians, Too Many Rules

Previously, Uber Eats had different search systems for different things: one for restaurants, one for dishes, and one for groceries. They were like separate librarians who didn't talk to each other.

• The Issue: If you searched for "tacos," the restaurant librarian might find a taco place, but the grocery librarian wouldn't know you might also want taco shells. It was slow, expensive to maintain, and often missed what you really wanted.

2. The Solution: The "Universal Translator" Librarian

The team built one single, super-smart librarian who understands everything.

• The Brain (Qwen2): They used a powerful AI brain (based on a model called Qwen2) that already knows a lot about the world. Think of this as hiring a librarian who has read every book in the world before they even started working at Uber Eats.
• The Two Towers: Imagine the librarian has two distinct roles:
  1. The Listener (Query Tower): This part listens to what you type ("I'm hungry for Italian").
  2. The Cataloger (Document Tower): This part knows everything about the millions of restaurants and items in the database.
  • Instead of reading every single book to find an answer, the librarian converts both your question and the books into secret codes (embeddings). If the codes match, the librarian knows it's a good match, even if the words are different!

3. The Training: Learning from Millions of Mistakes

You can't just hire a smart librarian and expect them to know Uber Eats immediately. They need to learn.

• Phase 1 (The Crowd): They showed the librarian hundreds of millions of examples of what people actually clicked on or added to their carts. This taught the librarian the general rules of "what people usually want."
• Phase 2 (The Tough Questions): Then, they used a "tough coach" (another AI) to find the hardest, trickiest questions where the librarian was confused. They practiced on these specifically so the librarian wouldn't make the same mistakes twice.

4. The Magic Trick: The "Matryoshka" Dolls

Here is the coolest part. The librarian creates a very long, detailed secret code (1,536 numbers long) for every item.

• The Problem: Storing and searching through these long codes for billions of items is slow and expensive, like trying to carry a giant encyclopedia everywhere.
• The Solution (MRL): They used a technique called Matryoshka Representation Learning. Think of a Russian nesting doll.
  • The whole doll is the full, detailed code (high quality, but big).
  • But, you can take off the outer layers, and the inner doll is still a perfect, smaller version of the same thing.
  • Why it matters: If you need a fast answer on a phone with a slow connection, the system uses the small inner doll (fast, cheap). If you need the absolute best answer for a big screen, it uses the full doll. One model does it all.

5. The Results: Faster, Smarter, and Cheaper

When they turned this new system on:

• Better Matches: They found the right restaurants and dishes much more often, even in different languages (Spanish, French, Japanese, etc.).
• Fewer Empty Searches: People stopped getting "No results found" messages.
• More Orders: Because people found what they wanted faster, they ordered more food.
• Cost Savings: By using the "smaller dolls" (truncated codes) and compressing the data (quantization), they saved a huge amount of money on computer storage and speed, without losing quality.

The Bottom Line

Uber Eats stopped using a bunch of clumsy, separate tools and built one unified, multilingual, super-smart search engine. It learns from real user behavior, adapts to your speed needs on the fly (like a shape-shifting robot), and helps you find your next meal much faster. It's like upgrading from a dusty card catalog to a psychic librarian who knows exactly what you're craving before you even finish typing.

Technical Summary

Here is a detailed technical summary of the paper "Scaling Multilingual Semantic Search in Uber Eats Delivery."

1. Problem Statement

Uber Eats faced significant challenges in its search infrastructure due to a fragmented legacy system. Previously, the platform relied on:

• Vertical-Specific Models: Separate BERT-based models for restaurants, dishes, and grocery/retail items.
• Lexical Search: Simple keyword matching that failed to capture user intent beyond exact string matches.
• Operational Complexity: Maintaining multiple models increased engineering overhead and limited generalization across languages and categories.

The goal was to build a unified, multilingual, and multi-vertical semantic retrieval system capable of handling stores, dishes, and grocery items with high recall, low latency, and operational efficiency.

2. Methodology

The authors developed a production-grade Two-Tower Architecture based on the Qwen2 (specifically gte-Qwen2-1.5B) Large Language Model (LLM).

A. Data and Input Representation

• Data Sources: Hundreds of millions of implicit feedback signals (clicks, "add-to-cart" events) from December 2023 to March 2025 across three verticals: Global Search (stores), In-Store Search (dishes), and Grocery & Retail (items).
• Structured JSON Input: Instead of plain text concatenation, queries and documents are formatted as structured JSON objects. This allows the model to leverage field names (e.g., store_name, cuisine_type) to disambiguate context and handle different verticals within a single schema.
• Contextual Enrichment: Inputs include search terms, location, and language to resolve cross-lingual ambiguities (e.g., "pan" in Spanish vs. English).

B. Model Architecture

• Backbone: Two unshared towers (Query Encoder and Document Encoder) based on the gte-Qwen2-1.5B decoder-only LLM.
• Pooling: Uses End-of-Sentence (EOS) token pooling to generate embeddings, which outperformed last-token and average pooling in ablation studies.
• Matryoshka Representation Learning (MRL): A core innovation where a single high-dimensional embedding (1536D) is trained such that its prefixes (e.g., 128D, 256D, 512D) are independently informative. This allows the system to serve different embedding sizes based on latency/cost requirements without retraining.

C. Training Strategy (Two-Stage)

1. Stage 1: Domain Adaptation (InfoNCE):
   • Trained on massive-scale click logs using InfoNCE loss with in-batch negatives.
   • Objective: Push positive query-document pairs closer while pushing all other in-batch items away.
2. Stage 2: Hard Negative Fine-Tuning (Triplet NCE):
   • Uses a curated dataset of "hard" examples (semantically similar but irrelevant items) identified via LLM-based oracle labeling and human review.
   • Objective: Triplet NCE loss to sharpen decision boundaries on difficult cases.
3. Optimization: Only the last three layers of the Qwen2 model are fine-tuned to prevent catastrophic forgetting of pre-trained linguistic knowledge.

D. Scoring and Serving

• Primary Scoring: Cosine similarity for efficient Approximate Nearest Neighbor (ANN) search.
• Secondary Scoring: Exploration of a learned Feed-Forward Neural (FFN) distance function for micro-re-ranking to capture non-linear relationships lost in linear cosine similarity.
• Indexing: Utilizes Uber's internal Lucene Plus indices with HNSW graphs, supporting both float32 (full precision) and int8 (quantized) vectors.

3. Key Contributions

1. Unified Multi-Vertical System: Successfully replaced fragmented legacy models with a single, scalable two-tower model handling restaurants, dishes, and groceries simultaneously.
2. Scalable Training Recipe: A robust pipeline combining large-scale InfoNCE domain adaptation with LLM-curated hard-negative Triplet NCE fine-tuning.
3. Flexible Embeddings via MRL: Implemented Matryoshka Representation Learning to support multiple embedding dimensions (128–1536) from a single model, enabling dynamic cost-latency trade-offs.
4. Production Innovation: Detailed insights into integrating LLMs into production search, including JSON input strategies, EOS pooling, and the use of quantized indices (INT8) with minimal performance loss.
5. Comprehensive Evaluation: Extensive offline and online validation across six global markets (US, UK, Japan, France, Mexico, Taiwan) and three verticals.

4. Results

Offline Performance

• Recall Gains: The final Two-Stage fine-tuned model achieved massive improvements over the legacy Mini-LM baseline:
  • +88% improvement in Store R@20 (high precision).
  • +31% improvement in Store R@200 (high recall).
  • Significant gains across all verticals (Dishes, Grocery) and languages.
• MRL Efficiency: Truncating the embedding from 1536D to 256D resulted in a negligible recall drop (0.002%) while reducing storage footprint to 16% and index cost to 23%.
• Quantization: Applying INT8 quantization to the 256D MRL cut reduced index size to 4% of the original with only a minor offline recall penalty, which was negligible in online settings.

Online Business Impact (A/B Testing)

• Conversion Rate (CVR): +0.44% lift in Intentful Search CVR.
• Search CoPSU: +0.54% lift in Orders/Clicks Per Search Unit.
• Zero-Result Rate: Reduced by 69.10%, significantly improving coverage in multilingual and multi-vertical scenarios.
• Latency: Retrieval latency remained flat, though end-to-end search latency increased by 18% due to a larger candidate set being processed by the downstream ranking model.

5. Significance and Future Work

This paper demonstrates that Large Language Models (LLMs) can be effectively adapted for large-scale, multi-vertical e-commerce search, surpassing traditional BERT-based and lexical approaches. The integration of MRL offers a novel solution to the operational complexity of managing multiple model sizes, while the two-stage training (InfoNCE + Triplet NCE) proves essential for handling hard retrieval cases.

Future Directions:

• Scaling to larger backbones (7B/14B parameters).
• Incorporating multi-modal signals (images) into the document tower.
• Moving toward fully isolated indices per model version for safer rollouts.
• Developing multi-objective learning frameworks to jointly optimize retrieval relevance and downstream conversion (pCVR).

In summary, Uber Eats has successfully transitioned to a state-of-the-art semantic search system that unifies diverse product categories, supports global languages, and optimizes for both retrieval quality and infrastructure cost.