💬 NLP

Structure-Guided Entity Resolution: Fine-Tuning LLMs for Robust Name Matching in Complex Linguistic Contexts

This paper introduces Structure-Guided Entity Resolution (SGER), a two-phase curriculum fine-tuning framework for Large Language Models that achieves state-of-the-art accuracy in matching complex, multilingual personal names and is currently deployed at scale for KYC compliance on Dream11's platform.

Original authors: Shivam Chourasia, Hitesh Kapoor, Nilesh Patil

Published 2026-05-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Shivam Chourasia, Hitesh Kapoor, Nilesh Patil

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to match two people's names, but the names are written in a chaotic, messy way. One person might be listed as "Rajesh Kumar," another as "Kumar Rajesh," a third as "RajeshKumar" (no space), and a fourth as "Rajeshbhai" (with a friendly title added). In a country as diverse as India, where names change based on region, language, and even how a clerk typed them, this is a nightmare for computers.

This paper introduces a new system called SGER (Structure-Guided Entity Resolution) that solves this problem with a clever two-step training method. Here is how it works, explained simply:

The Problem: The "Messy Name" Puzzle

In the world of "Know Your Customer" (KYC) checks—where companies verify who you are before letting you play games or open accounts—computers often fail.

The Old Way: Traditional computers use simple rules, like counting how many letters are different between two names. It's like trying to solve a puzzle by only looking at the color of the pieces, ignoring the shape. If someone writes "Shubham" instead of "Subham," the computer gets confused.
The New Way (LLMs): Large Language Models (AI) are smart, but if you just ask them "Are these two names the same person?" they sometimes guess wrong because they haven't learned the grammar of names first. They try to learn the structure and the answer at the same time, which is like asking a student to write a thesis and solve a math problem simultaneously.

The Solution: A Two-Phase "School" for AI

The authors created a curriculum (a lesson plan) to teach their AI model, based on a system called Llama 3, in two distinct phases. Think of it like training a new employee:

Phase 1: The "Name Architect" Class
Before the AI tries to match names, it is taught to break them apart.

The Task: You give the AI a messy name like "Kirtan Singh Rathore."
The Lesson: The AI must output a neat, structured list (like a JSON file) that says: First Name: Kirtan, Middle Name: Singh, Last Name: Rathore.
The Analogy: Imagine teaching a child to take a jumbled pile of Lego bricks and sort them into "wheels," "windows," and "walls" before trying to build a car. The AI learns that "Singh" is often a middle name or a family name, and "Rathore" is the family name, regardless of the order they appear in.

Phase 2: The "Detective" Class
Now that the AI understands how names are built, it is promoted to the detective role.

The Task: You give it two names (e.g., "Rajeshk" and "Rajesh Kumar") and ask, "Are these the same person?"
The Advantage: Because the AI already knows how to deconstruct names from Phase 1, it doesn't have to guess the structure while making the decision. It can focus purely on the match.
The Result: It becomes incredibly accurate at spotting that "Rajeshk" is just a typo or abbreviation of "Rajesh Kumar."

The Results: A Super-Accurate System

The team tested this system on 50,000 real-world examples from India, a place known for having some of the most complex and varied naming conventions in the world.

Old Methods: Got about 57% to 85% accuracy.
Standard AI (without the two-step training): Got about 91% accuracy.
SGER (The New System): Achieved 99.02% accuracy.

This means the system is almost perfect at telling if two names belong to the same person, even when the names are misspelled, swapped, or missing spaces.

Real-World Impact: Dream11

This isn't just a theory; it's already working. The system is deployed at Dream11, the world's largest fantasy sports platform, which serves over 250 million users.

Before: When the computer wasn't sure, it had to send the case to a human to check manually. This was slow and expensive.
After: The AI handles almost everything automatically.
The Benefit: The company saves over $500,000 a year because they don't need to pay humans to check these cases anymore, and legitimate users get verified instantly without waiting.

Summary

The paper argues that to make AI good at a specific, messy job (like matching names in India), you shouldn't just throw data at it. Instead, you should teach it the rules of the game (how names are structured) first, and then teach it how to play the game (match the names). This "curriculum" approach turned a good AI into a near-perfect one, solving a massive headache for a global company.

Technical Summary: Structure-Guided Entity Resolution (SGER)

Problem Statement

Entity Resolution (ER) in linguistically and culturally complex environments, particularly within Indian identity data, presents a significant challenge for Know Your Cus- tomer (KYC) compliance. Unlike stable keys, names in these contexts vary due to diverse cultural conventions, inconsistent transliteration across scripts (e.g., Devanagari to Latin), and frequent data entry errors (typos, merged tokens, missing whitespace). Traditional methods like edit distance, Jaro-Winkler, and phonetic encodings struggle with these structural ambiguities, leading to high rates of false positives and false negatives. While Large Language Models (LLMs) offer promise, standard fine-tuning often forces models to simultaneously learn name structure and classification boundaries, resulting in suboptimal performance in domains where structural patterns are strong but implicit.

Methodology: Structure-Guided Entity Resolution (SGER)

The authors propose SGER, a novel framework that fine-tunes an LLM (specifically Meta's Llama 3 8B) using a two-phase curriculum learning strategy. This approach decouples structural understanding from the binary matching task to build a robust internal representation of names.

System Architecture

Base Model: Llama 3 8B, chosen for its balance of accuracy, cost-efficiency, and open-source reproducibility.
Training Mechanism: Supervised Fine-Tuning (SFT) with Low-Rank Adaptation (LoRA) (Rank 4, Alpha 8) using mixed precision (fp16).

Phase 1: Name Structure Understanding

Objective: Teach the model the "internal grammar" of names.
Task: Map a raw name string (e.g., "Kirtan Singh Rathore") to a structured JSON object containing first_name, middle_name, and last_name.
Data: A dataset of ~10,000 manually annotated Indian names, augmented with synthetic variations to cover regional and linguistic diversity.
Outcome: The model learns to identify name components and handle structural variations (e.g., recognizing "SinghKirtan" as a merged form) before attempting to match pairs.

Phase 2: Binary Name Matching

Objective: Perform the downstream task of determining if two name variants refer to the same individual.
Initialization: Training begins from the Phase 1 checkpoint.
Task: Binary classification ("Yes" or "No") based on a pair of names.
Prompt Design: Includes instructions and few-shot examples crafted with risk operations teams to reflect real-world ambiguities (abbreviations, reordered components, honorifics).
Data Augmentation: To ensure robustness, the training data (originally 20,000 pairs) was expanded to over 50,000 examples using three strategies inspired by computer vision:
1. Pair Swapping: Ensures order invariance.
2. Component Permutation: Shuffles name parts to create valid but novel structural examples.
3. Random Space Removal: Simulates data entry errors by merging tokens (e.g., "Kirtan Singh" $\to$ "KirtanSingh").

Key Contributions

Curriculum-Based Framework: The paper introduces the first empirical validation of a two-phase curriculum learning framework tailored specifically for name-based entity resolution in complex, real-world settings. It demonstrates that separating structural learning from classification yields superior performance compared to single-stage fine-tuning.
State-of-the-Art Performance: SGER achieves 99.02% accuracy and an F1 score of 0.994 on a held-out set of 50,000 real-world Indian KYC pairs.
Production Deployment: The system is fully deployed in the production pipeline of Dream11 (serving 250M+ users), processing up to 10,000 requests per minute with a P99 latency of 120 milliseconds.

Results and Comparative Analysis

The authors evaluated SGER against five baselines on a disjoint validation set:

Levenshtein Similarity: 0.726 F1 (Struggles with structural variations).
BERT (Fine-Tuned): 0.806 F1.
GPT-4o (Few-Shot): 0.911 F1.
LLM (Few-Shot, Llama 3 8B): 0.839 F1.
LLM (Single-Stage SFT): 0.946 F1.
LLM (SFT + Augmentation): 0.973 F1.
SGER (Two-Phase + Augmentation): 0.994 F1.

Ablation Findings:

Data Augmentation: Increased F1 from 0.946 to 0.973 by preventing the model from memorizing surface patterns and improving generalization to "long tail" distortions.
Curriculum Learning: The addition of Phase 1 structural pre-training provided the final boost to 0.994, particularly stabilizing performance on pairs with multiple simultaneous distortions (e.g., abbreviated, reordered, and merged names).

Error Analysis:
The remaining errors (<1%) primarily stem from:

Ambiguous Phonetic Cases: Names that are phonetically similar but spelled differently due to inconsistent transliteration (e.g., "Saurabh" vs. "Sorab").
Complex Compound Errors: Cases involving simultaneous whitespace loss, order inversion, and OCR corruption (e.g., "SubhamKshr" vs. "KishoreShubham").

Significance and Impact

The paper claims that SGER sets a new standard for entity resolution in complex, multilingual domains by demonstrating that curriculum-guided training enables robust, high-precision matching at scale.

Operational Impact: The deployment has eliminated the need for manual review of ambiguous cases (previously routed to third-party vendors), resulting in over $500,000 in annual cost savings.
Compliance: The system achieves 99.95% precision, significantly reducing false links and the associated regulatory risks in KYC workflows.
Scalability: The approach proves that specialized entity resolution tasks do not necessarily require massive models; a modest 8B parameter model, when guided by a structured curriculum, can achieve near-perfect accuracy.

The authors conclude that while focused on Indian KYC data, this methodology is broadly applicable to other settings characterized by rich linguistic diversity, irregular naming conventions, and noisy inputs, such as global e-commerce and cross-border fraud detection.