arXiv:2603.12824cs.IRcs.CV2026-03中稿 · EMNLP被引 1

将大模型视觉检索能力压缩到69M小文本模型,实现高速低耗的文档检索。

NanoVDR: Distilling a 2B Vision-Language Retriever into a 70M Text-Only Encoder for Visual Document Retrieval

  • 用20亿参数大模型离线索引文档,70M小模型仅处理查询文本。
  • 新对齐方法使小模型保留95.1%大模型性能,推理速度提升50倍。
  • 适用于低资源部署场景,尤其适合多语言文档快速检索。

基于视觉语言模型(VLM)的检索器显著提升了视觉文档检索(VDR)性能,但需在索引和查询时使用相同的大规模编码器,导致高延迟与高显存依赖,即使对纯文本查询亦然。我们发现该设计过于对称:文档具视觉复杂性,需强视觉理解;而查询仅为短文本。NanoVDR利用查询-文档不对称性,解耦编码路径:冻结的20亿参数教师模型离线索引文档,蒸馏出仅6900万参数的小型文本编码器实时处理查询。关键设计是蒸馏目标:在三种骨干网络、22个ViDoRe基准数据集上系统对比六种目标后发现,基于查询文本的点对点余弦对齐优于排序与对比学习,且仅需预缓存教师端查询嵌入,训练时不需文档处理。此外,我们识别出跨语言迁移为性能瓶颈,通过引入机器翻译查询数据廉价缓解。最终,NanoVDR-S-Multi(DistilBERT,69M)保留教师模型95.1%性能,在v2/v3上超越DSE-Qwen2(20亿参数),参数量减少32倍,CPU查询延迟降低50倍,总训练成本低于13 GPU小时。

原文摘要 · Abstract (English)

Vision-Language Model (VLM) based retrievers have advanced visual document retrieval (VDR) to impressive quality. They require the same multi-billion parameter encoder for both document indexing and query encoding, incurring high latency and GPU dependence even for plain-text queries. We observe that this design is unnecessarily symmetric: documents are visually complex and demand strong visual understanding, whereas queries are just short text strings. NanoVDR exploits this query--document asymmetry by decoupling the two encoding paths: a frozen 2B VLM teacher indexes documents offline, while a distilled text-only student as small as 69M parameters encodes queries at inference. The key design choice is the distillation objective. Through systematic comparison of six objectives across three backbones and 22 ViDoRe benchmark datasets, we find that pointwise cosine alignment on query text consistently outperforms ranking-based and contrastive alternatives, while requiring only pre-cached teacher query embeddings and no document processing during training. Furthermore, we identify cross-lingual transfer as the primary performance bottleneck, and resolve it cheaply by augmenting training data with machine-translated queries. The resulting NanoVDR-S-Multi (DistilBERT, 69M) retains 95.1\% of teacher quality and outperforms DSE-Qwen2 (2B) on v2 and v3 with 32$\times$ fewer parameters and 50$\times$ lower CPU query latency, at a total training cost under 13 GPU-hours.

视觉检索模型压缩小模型多语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。