构建企业级检索基准，实现不重索引的高效查询优化
Succeeding at Scale: Enterprise Retrieval Benchmark Construction and Index-Preserving Query Adaptation for Multi-Tenant Search
- 用多模型融合+大模型判读自动构建技术问答检索数据集
- 仅微调查询编码器就能大幅提升检索效果，无需重新索引文档
- 适合需要快速迭代但无法重索引的企业级多租户检索系统
大规模多租户检索系统产生大量查询日志，但缺乏标注的相关性标签，导致大量‘暗数据’未被利用。同时，模型更新成本高：联合微调查询和文档编码器需全量重索引，这在拥有数千个独立索引的多租户场景中不可行。本文提出 DevRev-Search，一个基于全自动流程构建的技术客户支持段落检索基准。候选生成采用多种稀疏与密集检索器融合，再通过 LLM-as-a-Judge 进行一致性过滤与相关性标注。进一步研究并系统评估了保持索引不变的仅查询端适配策略——仅微调查询编码器，固定文档索引。在 DevRev-Search、SciFact 与 FiQA-2018 上的实验表明，参数高效的查询编码器微调实现了显著的质量-效率平衡，使可扩展且实用的企业级多租户检索成为可能。
原文摘要 · Abstract (English)
Large-scale multi-tenant retrieval systems generate extensive query logs but lack curated relevance labels for effective domain adaptation, resulting in substantial underutilized "dark data." This challenge is compounded by the high cost of model updates, as jointly fine-tuning query and document encoders requires full corpus re-indexing, which is impractical in multi-tenant settings with thousands of isolated indices. We introduce DevRev-Search, a passage retrieval benchmark for technical customer support built via a fully automated pipeline. Candidate generation uses fusion across diverse sparse and dense retrievers, followed by an LLM-as-a-Judge for consistency filtering and relevance labeling. We further study and systematically evaluate index-preserving query-only adaptation strategies that fine-tune only the query-encoder while keeping the document indices fixed. Experiments on DevRev-Search, SciFact, and FiQA-2018 show that parameter-efficient fine-tuning of the query encoder delivers a remarkable quality-efficiency trade-off, enabling scalable and practical enterprise multi-tenant retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。