Xuanwu: Evolving General Multimodal Models into an Industrial-Grade Foundation for Content Ecosystems

1. Problem Statement

While general Multimodal Large Language Models (MLLMs) like LLaVA and InternVL perform well on public benchmarks, they struggle in real-world industrial content moderation and adversarial scenarios. The authors identify three core challenges:

• Inference Cost & Catastrophic Forgetting: Very large models are too expensive for high-concurrency social platforms. Furthermore, direct domain adaptation often causes "catastrophic forgetting," where the model loses general world knowledge and long-tail capabilities.
• Insufficient Granularity: General models lack awareness of specific business contexts, such as regional regulations, community slang, and fine-grained legal/ethical boundaries required for high-precision moderation.
• Weak Adversarial Robustness: Mainstream models fail to detect policy-violating text hidden in low-resolution images, heavily distorted watermarks, or AIGC-generated forgeries, leading to low recall rates in adversarial settings.

2. Methodology

The authors propose Xuanwu VL-2B, a compact, industrial-grade foundation model designed to balance business specialization with general capability retention under a ~2B parameter budget.

A. Model Architecture

• Compact Design: The model adopts a "Minimalist Architecture with Stable Components" to ensure training stability and deployment efficiency.
  • Vision Encoder: InternViT-300M (selected over larger variants like AIMv2-Huge for better trade-offs between performance and cost).
  • Language Backbone: Qwen3 1.7B.
  • Connector: A lightweight 2-layer MLP projector (avoiding complex Q-Former or MoE structures).
• Dynamic High-Resolution Perception: To handle tiny text and distorted watermarks, the model uses a Dynamic Tiling & Resizing mechanism. It splits images into up to 12 local high-resolution tiles (448×448) plus a global thumbnail, then applies Pixel Unshuffle to reduce token counts, preserving fine-grained details without exploding computational costs.

B. Three-Stage Training Pipeline

The model is trained through a progressive pipeline to inject business knowledge while retaining general skills:

1. Pre-Training:
   • Modality Alignment: Aligns the MLP projector using 1.3M high-quality caption pairs.
   • General Knowledge: Full-parameter joint training on 17.33M image-text pairs to build a strong visual-language foundation.
2. Mid-Training (Business Injection):
   • Trained on ~2.8M samples including:
     • Base Retention: 10% of pre-training data to prevent forgetting.
     • Instruction Following: Enhanced with AutoIF-Instruct and Tulu-3 data.
     • Content Moderation: Positive/negative moderation data and OCR annotations.
     • Adversarial SPAM: Synthetic and human-annotated spam/adversarial samples.
   • Data Curation: Uses K-Means clustering and LLM judges to ensure distributional balance and quality, mitigating catastrophic forgetting.
3. Post-Training (Alignment):
   • Supervised Fine-Tuning (SFT): Uses ~8.4M high-fidelity samples with structured Chain-of-Thought (CoT) reasoning ("Observe → Reason → Conclude") to teach explicit moderation rules and output formats.
   • Reinforcement Learning (RL): Employs GRPO (Group Relative Policy Optimization) with a composite reward function combining classification accuracy, format compliance, and OCR alignment (measuring character-level matching of violating text).

3. Key Contributions

1. Business-Oriented Evaluation System: A comprehensive framework evaluating content understanding, moderation (7 categories), and adversarial defense (8 attack types), specifically tailored for industrial scenarios.
2. Three-Stage Training Strategy: A methodology that successfully balances business performance gains with the retention of general capabilities through data refinement and progressive alignment.
3. Compact 2B Architecture: Demonstrates that a ~2B parameter model (InternViT-300M + Qwen3 1.7B) can outperform larger open-source models and rival commercial giants in specific adversarial tasks while maintaining low deployment costs.
4. Advanced Data Curation & RL: Implementation of high-fidelity SFT data construction and GRPO-based alignment specifically optimized for adversarial OCR and spam detection.

4. Experimental Results

Xuanwu VL-2B was evaluated against InternVL 3.0/3.5 (2B) and the commercial model Gemini-2.5-Pro.

• General Capabilities:
  • Achieved an average score of 67.90 on 7 OpenCompass multimodal benchmarks, outperforming InternVL 3.5 2B (64.27).
  • Maintained competitive text-only performance (Average-9: 58.38), though with slight drops in specific math/code tasks due to business specialization.
• Content Moderation:
  • Achieved an average recall of 94.38% across 7 independent business moderation tasks (vs. 47.98% for InternVL 3.5 2B).
  • Showed massive improvements in long-tail categories like "high-risk" (97.99% vs 7.93%) and "illegal" (92.19% vs 12.93%).
• Adversarial OCR & Defense:
  • Achieved a weighted overall recall of 82.82% on challenging adversarial OCR scenarios (distorted text, watermarks, AIGC forgeries).
  • Outperformed Gemini-2.5-Pro (76.72%) and significantly surpassed open-source baselines (~64.7%).
  • Notable gains were seen in handling AIGC fused images, noise interference, and heavy watermarks.

5. Significance

This paper demonstrates that industrial-grade multimodal foundations do not require massive parameter counts to be effective. By focusing on:

1. Architecture Efficiency: Using stable, dense components rather than complex MoE.
2. Data-Centric Training: Rigorous curation and a three-stage pipeline to balance specialization and generalization.
3. Task-Specific Alignment: Using GRPO and CoT to handle complex adversarial logic.

The authors successfully built a model that is cost-effective for large-scale deployment while achieving state-of-the-art performance in content moderation and adversarial defense, surpassing even leading commercial models in specific zero-shot adversarial tasks. This provides a blueprint for developing specialized foundation models for safety-critical content ecosystems.