DistilVDR: A Compact End-to-End Visual Document Retriever via Dual-Student Distillation

Technical Summary of DistilVDR

Problem Statement
Visual Document Retrieval (VDR) is currently dominated by massive vision-language models (ranging from 2 to 8 billion parameters) that encode document pages as images to avoid OCR errors and preserve layout. While effective, these models are prohibitively expensive to deploy: they require significant GPU memory for indexing, incur high latency during scoring, and generate massive index footprints, particularly when using multi-vector architectures. Existing compression strategies face a trade-off: training smaller multi-vector encoders from scratch preserves quality but retains the high indexing and scoring costs of multi-vector systems, while distilling only the query side leaves the document encoder at the teacher's massive scale. There is a lack of compact, end-to-end single-vector retrievers that achieve high quality without the overhead of multi-vector interaction or massive parameter counts.

Methodology
The authors propose DistilVDR, a 524 M parameter end-to-end single-vector VDR system. The approach relies on dual-student distillation from a single, frozen 8 B vision-language teacher (Qwen3-VL-Embedding-8B).

• Architecture: The system employs an asymmetric, encoder-only student design tailored to VDR's input asymmetry (text queries vs. image documents).
  • Query Encoder: A lightweight 70 M parameter text-only encoder based on DistilBERT-base. It processes text queries prefixed with the same instruction string used to train the teacher.
  • Document Encoder: A heavier 454 M parameter encoder combining a 300 M InternViT visual encoder and a 150 M ModernBERT-base text backbone. It processes document images via a dynamic tiling strategy that adapts the grid layout to the page's aspect ratio while capping the total visual token count to fit the backbone's context window. Visual tokens are projected into the text backbone's embedding space and processed with bidirectional attention.
• Training Objective: Both student encoders are trained independently to regress onto the cached 4096-dimensional embeddings produced by the frozen teacher. The loss function is a point-wise cosine alignment loss ($L = 1 - \langle \text{student}, \text{teacher} \rangle$).
  • Crucially, the student objective requires no relevance labels, no negative sampling, and no contrastive terms. The supervision is derived entirely from the teacher's embedding space, which was pre-trained with relevance supervision.
  • The two students are decoupled during training, allowing for parallel optimization.
• Variants: Two deployment variants are released, differing only in the visual token budget (tile count) for the document encoder:
  • DistilVDR-HiRes: Uses up to 6 tiles plus a thumbnail (max 7,168 tokens).
  • DistilVDR-Fast: Uses up to 2 tiles plus a thumbnail (max 3,072 tokens).

Key Results
Evaluated on the full 22-dataset ViDoRe suite (v1, v2, and v3), DistilVDR demonstrates significant improvements in quality-efficiency trade-offs:

• Retrieval Quality: DistilVDR-HiRes achieves an average NDCG@5 of 61.74 on ViDoRe, retaining 86.9% of the 8 B teacher's performance (71.05). It outperforms every reproduced sub-1 B baseline, including multi-vector models like colSmol-500M (53.01) and ColModernVBERT (42.46). On the high-resolution-sensitive v3 benchmark, it leads all sub-1 B baselines by a wide margin (47.07 vs. 33.52 for the next best). DistilVDR-Fast achieves 59.98 average NDCG@5 with a 3× smaller visual token budget.
• Efficiency:
  • Indexing: DistilVDR indexes a corpus an order of magnitude faster than the 8 B teacher and significantly faster than multi-vector baselines.
  • Storage: The single-vector index is 15.6 times smaller than the strongest sub-1 B multi-vector baseline (16.4 GB vs. 256 GB per million documents).
  • Scoring: Scoring latency is reduced by two orders of magnitude compared to multi-vector MaxSim scoring (9.6 ms for 10k documents vs. ~1.1–3.2 s).
  • Throughput: DistilVDR-Fast achieves 99.04 docs/sec, roughly 18× faster than the 8 B teacher.

Significance and Claims
The paper claims that DistilVDR successfully closes the gap between the high quality of massive models and the deployment efficiency of compact systems. By distilling both the query and document encoders from a single teacher under a cosine alignment loss, the authors demonstrate that a compact, single-vector retriever can be trained end-to-end without contrastive learning or negative mining.

The authors position DistilVDR as a superior operating point at the 500 M scale, dominating the "from-scratch" small multi-vector route in both retrieval quality and all deployment metrics (latency, VRAM, index size). They emphasize that the system's success relies on the teacher's pre-trained embedding space, which transfers relevance knowledge without requiring the student to learn it from scratch via contrastive objectives.

Limitations and Scope
The authors note several limitations:

• Teacher Dependency: The student is bounded by the teacher's capabilities; systematic weaknesses in the 8 B teacher propagate to DistilVDR.
• Experimental Scope: Comparisons are made against official public releases rather than retrained baselines with matched compute budgets. The paper does not sweep teacher scales or families.
• Performance Gap: While competitive with 2–3 B models, DistilVDR still trails the strongest 4–8 B multi-vector models by 7–10 points, particularly on the complex v3 benchmark.
• Content Adaptivity: The tiling budget is fixed at release time based on aspect ratio, not page content complexity.
• Query Scope: The query encoder is text-only and trained on English and Latin-script languages; it does not support image-conditioned queries or non-Latin scripts.
• Compression: The reported index footprint assumes uncompressed float32 vectors; further quantization techniques were not tested.