arXiv:2603.26683cs.IRcs.AI2026-03

通过生成多个查询变体,提升多模态文档检索的准确性和鲁棒性。

LITTA: Late-Interaction and Test-Time Alignment for Visually-Grounded Multimodal Retrieval

  • 用大模型生成查询变体,通过融合理解增强检索覆盖。
  • 在三个领域中,顶K准确率、召回率和MRR均显著提升。
  • 无需重训练,可灵活调节查询数量以平衡效率与精度。

从包含丰富视觉内容的文档(如教科书、技术报告、手册)中检索相关证据极具挑战,原因包括长上下文、复杂版式以及用户问题与支持页面间弱词汇重叠。我们提出LITTA,一种以查询扩展为核心的检索框架,用于证据页检索,可在不重新训练检索器的前提下提升多模态文档检索性能。给定用户查询,LITTA利用大语言模型生成互补的查询变体,并使用冻结的视觉检索器对每个变体进行候选页检索,采用延迟交互评分策略。通过倒数排名融合聚合各变体结果,提升证据覆盖范围并降低对单一提问方式的敏感性。该测试时策略显著增强检索鲁棒性,且兼容现有多模态嵌入索引。我们在计算机科学、制药和工业手册三个领域评估了LITTA,多查询检索在所有指标上均优于单查询检索,尤其在视觉与语义变化较大的领域收益显著。通过调整查询变体数量,可直接控制精度-效率权衡,适用于有延迟约束的部署场景。结果表明,查询扩展是一种简单而有效的提升视觉接地多模态检索的方法。

原文摘要 · Abstract (English)

Retrieving relevant evidence from visually rich documents such as textbooks, technical reports, and manuals is challenging due to long context, complex layouts, and weak lexical overlap between user questions and supporting pages. We propose LITTA, a query-expansion-centric retrieval framework for evidence page retrieval that improves multimodal document retrieval without retriever retraining. Given a user query, LITTA generates complementary query variants using a large language model and retrieves candidate pages for each variant using a frozen vision retriever with late-interaction scoring. Candidates from expanded queries are then aggregated through reciprocal rank fusion to improve evidence coverage and reduce sensitivity to any single phrasing. This simple test-time strategy significantly improves retrieval robustness while remaining compatible with existing multimodal embedding indices. We evaluate LITTA on visually grounded document retrieval tasks across three domains: computer science, pharmaceuticals, and industrial manuals. Multi-query retrieval consistently improves top-k accuracy, recall, and MRR compared to single-query retrieval, with particularly large gains in domains with high visual and semantic variability. Moreover, the accuracy-efficiency trade-off is directly controllable by the number of query variants, making LITTA practical for deployment under latency constraints. These results demonstrate that query expansion provides a simple yet effective mechanism for improving visually grounded multimodal retrieval.

多模态检索查询扩展视觉接地测试时优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。