💬 NLP

From Unstructured to Structured: LLM-Guided Attribute Graphs for Entity Search and Ranking

This paper proposes a two-stage LLM-guided framework that constructs structured attribute graphs from unstructured e-commerce data to enable efficient, high-precision entity search and ranking without requiring training data.

Original authors: Yilun Zhu, Nikhita Vedula, Shervin Malmasi

Published 2026-05-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yilun Zhu, Nikhita Vedula, Shervin Malmasi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are shopping online for a specific item, like a "red, wireless, noise-canceling headphone." You want the system to show you other headphones that are almost exactly the same, not just any headphones that happen to be red.

The paper describes a new way to build a search engine that does this much better than before. Here is how it works, broken down into simple concepts:

The Problem: The "Messy Desk" vs. The "Organized Filing Cabinet"

Traditionally, search engines look at product descriptions like a messy desk. They read the whole paragraph of text (e.g., "This amazing red headphone has wireless capabilities and cancels noise..."). Because the text is unstructured and varies wildly from product to product, the computer often misses the specific details. It might think two products are similar just because they both use the word "red," even if one is a tiny earbud and the other is a giant over-ear headset.

The Solution: A Two-Step "Translator and Judge" System

The authors created a system that acts like a two-step process: first, a Translator, and second, a Judge.

Step 1: The Translator (Offline Stage)

Before anyone even searches, the system takes all the messy product descriptions and uses a powerful AI (a Large Language Model) to act as a translator.

What it does: It reads the messy text and pulls out specific, organized facts, like a librarian organizing books. It creates a "schema" (a checklist) for each type of product.
- Example: For a TV, it looks for "Screen Size" and "Resolution." For a shirt, it looks for "Material" and "Size."
The Result: It turns the messy paragraph into a clean, structured list of facts (e.g., Color: Red, Type: Wireless, Feature: Noise Canceling).
The Graph: It then connects these facts into a giant web (a graph). Imagine a spiderweb where the "products" are one set of nodes and the "attributes" (like "Red" or "Wireless") are the other nodes. This creates a clear map of how products relate to each other based on their specific features, not just their words.

Step 2: The Judge (Online Stage)

When you actually search for a product, the system doesn't ask the AI to read the whole messy description again.

The Shortcut: It first finds a small group of potential matches (candidates) using a fast, standard search.
The Comparison: It then asks the AI to act as a judge. Instead of reading 700 words of text, the AI looks at the clean, organized lists created in Step 1.
The Analogy: Imagine you are comparing two resumes.
- Old Way: You read the whole story of their life for every single candidate. It takes forever and you might miss details.
- New Way: You have a spreadsheet where every candidate's "Years of Experience," "Degree," and "Salary" are in the same column. You just look down the column to compare them instantly.

Why This is a Big Deal

The paper claims this method is a game-changer for three main reasons:

It's Smarter: By comparing specific facts (like "50-inch screen" vs. "55-inch screen") rather than guessing from text, the system finds better matches. In tests, it improved the accuracy of finding the right products by over 5%.
It's Faster and Cheaper: Because the AI doesn't have to read the whole messy description, it processes much less information. The paper says this reduced the amount of data the AI has to "read" by 57%.
- Analogy: It's like sending a 1-page summary instead of a 300-page novel to a lawyer. The lawyer can give you an answer much faster and it costs less.
It Works Without Training: The system is "zero-shot," meaning it doesn't need to be taught with thousands of examples of "good" and "bad" matches. It just uses its general intelligence to understand the structure of the data immediately.

Real-World Impact

The authors have already deployed a version of this at a major e-commerce company. They found that it not only finds better products for users (making them happier) but also saves money on computing power because the AI works so much more efficiently.

In short, they turned a chaotic library of product descriptions into a perfectly organized database, allowing the AI to make fair, precise, and fast comparisons between products.

1. Problem Statement

The paper addresses the challenge of Entity Search in e-commerce, specifically the task of finding products most similar to a query product (e.g., finding substitutes).

Core Difficulty: Product information is typically unstructured (titles, descriptions, bullet points), making it difficult to perform systematic, attribute-level comparisons.
Limitations of Existing Methods:
- Traditional Embeddings: Dense semantic models often fail to capture nuanced, context-specific attribute relevance (e.g., distinguishing between "screen size" for TVs vs. "material" for clothing).
- Direct LLM Ranking: Feeding raw, unstructured text directly into Large Language Models (LLMs) for ranking is costly, introduces high latency, consumes excessive tokens, and struggles to enforce hard constraints on critical attributes.
- Cold Start & Dynamic Categories: Traditional collaborative filtering fails with new products, and attribute sets are dynamic, requiring constant adaptation to new categories.

2. Methodology

The authors propose a two-stage hybrid system that decouples knowledge construction from ranking, utilizing an Attribute-Based Graph Ranker.

Stage 1: Offline Attribute Graph Construction

This stage enriches the product catalog by converting unstructured text into a structured, reusable graph.

Attribute Definition:
- The system defines hierarchical attribute schemas (Super-category and Sub-category levels) using an LLM (Claude 3.5 Sonnet v2).
- It generates broad attributes (e.g., Brand, Dimensions) and specific ones (e.g., Camera Resolution for Smartphones).
- Human-in-the-loop curation (clustering/deduplication) ensures naming consistency across 61 super-categories and 4,940 sub-categories.
Attribute Extraction:
- The LLM processes product text (title, description, bullets) to extract structured Attribute-Value (AV) pairs based on the predefined schemas.
- It also identifies product-specific attributes not in the initial schema.
Graph Construction:
- A bipartite graph is constructed where nodes represent Products and Attributes.
- Edges connect products to attributes, labeled with the specific Value.
- Optimization: Attributes are nodes, and values are edge labels to prevent the graph from becoming intractably large (avoiding a unique node for every unique value).

Stage 2: Online Graph-Aware LLM Ranking

This stage performs the actual search and ranking using the structured graph.

Candidate Generation:
- A query entity is received.
- A dense retrieval model (bge-m3) encodes the query and catalog items to retrieve the top- $K$ candidates (e.g., top 50) based on cosine similarity of concatenated text.
Graph-Aware Ranking:
- Instead of feeding raw text to the LLM, the system constructs a local sub-graph containing the query entity, the candidate entities, and their associated attribute nodes.
- The LLM performs zero-shot, list-wise reasoning by comparing the attribute values of the query against the candidates within this sub-graph.
- Scoring: The LLM implicitly traverses the graph edges to assess similarity, assigning a score (0–100) based on how well attribute values align.
- Efficiency: The input to the LLM is sanitized, structured JSON (AV pairs) rather than free text, reducing token usage and mitigating adversarial risks.

3. Key Contributions

LLM-Based Attribute Graph Construction: A pipeline that automatically defines category-aware schemas and extracts structured AV pairs from unstructured text, creating a scalable, reusable attribute graph.
Graph-Aware LLM Ranking: A novel ranking mechanism where the LLM reasons over structured attribute graphs rather than raw text. This enables explicit attribute-value comparisons, improving precision and interpretability.
Decoupling Strategy: Separating the heavy lifting of knowledge construction (offline) from the inference process (online) allows for efficient, low-latency ranking without re-training models.
Scalable Deployment: The system is designed for industrial scale, handling millions of products across diverse categories with zero-shot capabilities.

4. Experimental Results

The system was evaluated on the Amazon Shopping Queries dataset (469k products) using both human evaluation and LLM-based evaluation (Nova Pro).

Performance Metrics:
- Precision@5: Improved from 39.52% (Dense Retrieval + Raw LLM Ranker) to 47.14% (Graph Ranker).
- Mean Reciprocal Rank (MRR): Increased from 60.00% to 63.97%.
- Mean Average Precision (mAP): Improved from 57.99% to 60.42%.
- nDCG@1: Improved from 75.27% to 79.96% (at strict similarity threshold).
Efficiency Gains:
- Token Reduction: Structured attributes reduced per-product token usage by 57% (approx. 300 tokens vs. 700 for raw text).
- Latency: Achieved approximately 250ms faster inference per request compared to raw-text ranking.
Extraction Accuracy: The attribute extraction step achieved an 83.47% F1 score on a 20k sample set, with hallucinations minimized via schema constraints.
Generalization: The system demonstrated robust performance across diverse product categories without requiring task-specific training data (zero-shot).

5. Significance and Impact

Cost and Latency Optimization: By reducing token consumption by 57%, the approach makes LLM-based ranking economically viable for high-volume e-commerce applications.
Improved Relevance: Moving from semantic "bag-of-words" similarity to explicit attribute matching allows the system to handle nuanced product differences (e.g., matching a 5-piece security kit to another 5-piece kit, rather than just "security system").
Real-World Deployment: The system is already deployed in a leading global e-commerce service, serving as input for downstream applications (pricing, availability) and training data for internal embedding models.
Interpretability: Unlike black-box dense embeddings, the graph-based approach provides transparent reasoning based on specific attribute matches, which is crucial for user trust and business rule enforcement.

In conclusion, this paper demonstrates that combining structured attribute extraction with LLM reasoning creates a superior entity search system that balances high precision, low latency, and scalability, outperforming both traditional retrieval methods and direct text-based LLM ranking.