SIA: A Synthesize-Inject-Align Framework for Knowledge-Grounded and Secure E-commerce Search LLMs with Industrial Deployment

1. Problem Statement

The deployment of Large Language Models (LLMs) in industrial e-commerce search faces two critical bottlenecks that hinder their adoption on platforms like JD.com:

• Knowledge Hallucination: General-purpose LLMs lack deep, dynamic encoding of fine-grained e-commerce knowledge (e.g., SKU relations, real-time inventory, user behavior logs). This leads to factual errors and "hallucinations" when handling specific product queries.
• Security and Compliance Vulnerabilities: Standard safety guardrails are often insufficient for the e-commerce domain. Models are susceptible to "jailbreak" attacks that could induce them to recommend prohibited goods, violate regulations, or expose sensitive data. Existing safety datasets lack coverage of domain-specific risks.

Current solutions like Retrieval-Augmented Generation (RAG) introduce latency, while standard fine-tuning (e.g., LoRA) often struggles to encode complex knowledge systems or suffers from "catastrophic forgetting" of general capabilities.

2. Methodology: The SIA Framework

The authors propose SIA (Synthesize-Inject-Align), an end-to-end framework designed to build a knowledge-grounded and secure e-commerce search LLM. The framework consists of three sequential stages:

Stage 1: Synthesize (Data Synthesis)

Instead of using raw data, the authors synthesize high-quality natural language corpora from heterogeneous sources:

• Context-Aware Knowledge Transformation: Combines structured Knowledge Graphs (KGs) and unstructured behavioral logs (reviews, Q&A) using an LLM (Deepseek-R1) to generate fluent, information-dense text descriptions. This includes product descriptions, attribute Q&A, comparisons, and review summaries.
• Chain-of-Thought (CoT) Synthesis: Mines real user session logs (query corrections, click-purchase paths) to generate instruction data with explicit reasoning chains, enhancing the model's multi-step decision-making capabilities.
• Safety-Centric Data Construction: Systematically defines high-risk e-commerce topics (e.g., illegal items, political sensitivity) and synthesizes a "trifecta" of defensive data:
  1. Safe Q&A pairs for standard compliance.
  2. Jailbreak prompts & defense examples (simulating 20+ attack vectors like role-playing, obfuscation, and logic exploitation).
  3. Boundary case generation for cautious decision-making.

Stage 2: Inject (Knowledge Injection via Parameter Expansion)

To inject massive domain knowledge without destroying general capabilities, the authors employ Depth Up-Scaling:

• Architecture Modification: New Transformer layers are inserted between existing blocks.
• Initialization & Learning Strategy:
  • Residual Identity Mapping: New layers are initialized to minimize disruption to pre-trained weights.
  • Hierarchical Learning Rates: New layers use high learning rates for rapid domain knowledge acquisition; original top layers use medium rates for adaptation; original lower/middle layers use very low or frozen rates to preserve general linguistic representations.
• Mitigating Forgetting: A Curriculum Learning strategy is used, starting with simple high-safety samples and gradually increasing complexity. Crucially, a 6:4 mix of e-commerce and general-domain data is used during training to prevent catastrophic forgetting.

Stage 3: Align (Fine-grained Domain-Enhanced Alignment)

The model undergoes a dual-path alignment process:

• Task-Side Alignment (Multi-Task Instruction Tuning): Uses a constructed dataset (Ecom-MTIT) with a three-tiered instruction system:
  1. Simple: Atomic tasks (e.g., rewrite query).
  2. Composite: Multi-intent simulation.
  3. Constrained Complex: Strict business rule adherence.
• Safety-Side Alignment (Adversarial Training):
  • Stage 1: Supervised Fine-Tuning (SFT) on 10k manually curated safety samples.
  • Stage 2: "Red-Teaming" where a specialized model (SIA-Red) generates 10k deceptive jailbreak prompts to train the main model (SIA-Safety) to resist them.
• Data Mixing: The final alignment dataset blends E-commerce tasks (60%), General instructions (20%), and Safety-aligned instructions (20%) to balance specialization and generalization.

3. Key Contributions

1. Novel Data Synthesis: A method to translate structured KGs and logs into high-quality, reasoning-rich natural language, solving the "knowledge sparsity" problem at the source.
2. Efficient Injection via Depth Up-Scaling: A parameter-efficient pre-training technique that expands model depth to absorb domain knowledge while preserving general capabilities through hierarchical learning rates and data replay.
3. Dual-Path Alignment: A strategy combining multi-task instruction tuning with adversarial red-teaming specifically tailored for e-commerce risks, ensuring both high performance and robust safety.
4. Industrial Deployment: Successful deployment on JD.com, validating the framework's scalability and effectiveness in a real-world, high-concurrency environment.

4. Experimental Results

The framework was evaluated on SIA-15B (based on Qwen2.5-14B-Instruct, expanded to ~15B parameters).

• E-commerce Performance:
  • EKQA (Knowledge QA): Improved from 23.87 (Baseline) to 58.01.
  • PAU (Product Attribute Understanding): Improved from 50.65 to 91.72.
  • EQEC (Query Correction): Improved from 39.04 to 74.29.
  • Note: SIA-15B outperformed DeepSeek-V3 on e-commerce tasks.
• General Capability Preservation:
  • Scores on benchmarks like C-Eval, CMMLU, and MMLU remained stable or slightly improved, proving that catastrophic forgetting was effectively mitigated.
• Safety & Compliance:
  • On the JDSec benchmark (8 risk dimensions), SIA-15B achieved a significantly lower Attack Success Rate (ASR) of 13.65 compared to the baseline (16.68) and even outperformed GPT-4o (14.92) in several categories.
• A/B Testing (Online Deployment):
  • Search Query Suggestion: +1.339% lift in Click-Through Rate (CTR).
  • Query Correction: +10% improvement in F1-score.
  • Product Title Generation: +1.93% lift in Conversion Rate (CVR).
  • Review Summarization: +0.15% lift in Unit Conversion Rate (UCVR).
  • Security: Deployed across 90% of JD.com's risk control scenarios, showing substantial improvements in detecting phishing, AIGC content, and code security issues.

5. Significance

The SIA framework represents a significant advancement in Industrial LLM deployment. It moves beyond simple fine-tuning or RAG by addressing the fundamental trade-off between domain specialization and general intelligence, while simultaneously solving the safety compliance challenge specific to regulated industries like e-commerce.

By successfully deploying this framework on JD.com, the authors demonstrate that it is possible to build LLMs that are not only "smart" enough to understand complex product ecosystems but also "safe" enough to operate within strict legal and regulatory boundaries at a massive scale. The methodology offers a reproducible blueprint for other vertical domains requiring high knowledge density and rigorous safety standards.