零样本生成式检索框架,让模型通过自然语言指令实现跨任务通用检索。
ZeroGR: A Generalizable and Scalable Framework for Zero-Shot Generative Retrieval
- 用自然语言指令驱动文档编码与查询生成,统一多模态数据
- 在BEIR和MAIR上达到当前最优,零样本性能超越现有方法
- 适合需要快速适配新检索任务的研究者和开发者
生成式检索(GR)将信息检索重构为生成文档标识符(docids)的任务,支持端到端优化并可无缝集成生成式语言模型。尽管监督训练下进展显著,但GR在真实场景中常见的零样本检索任务上仍难以泛化。为此,我们提出零样本生成式检索框架ZeroGR,利用自然语言指令扩展GR在多种检索任务上的适用性。ZeroGR包含三个核心组件:(i) 基于语言模型的docid生成器,将异构文档(如文本、表格、代码)统一为语义有意义的docids;(ii) 指令微调的查询生成器,从自然语言任务描述生成多样化查询以增强语料索引;(iii) 反向退火解码策略,在docid生成中平衡精确率与召回率。此外,我们构建了目前最多样化的开源指令检索数据集OpenInstIR。研究发现,指令微调覆盖的任务越多,性能越优。在BEIR和MAIR基准上的大量实验表明,ZeroGR在多种检索任务中表现优异,成为当前GR方法中的新标杆。代码已开源:https://github.com/sunnweiwei/ZeroGR。
原文摘要 · Abstract (English)
Generative retrieval (GR) reformulates information retrieval (IR) by framing it as the generation of document identifiers (docids), thereby enabling end-to-end optimization and seamless integration with generative language models (LMs). Despite notable progress under supervised training, GR still struggles to generalize to zero-shot IR scenarios, which are prevalent in real-world applications. To tackle this challenge, we propose ZeroGR, a zero-shot generative retrieval framework that uses natural language instructions to extend GR across a wide range of IR tasks. Specifically, ZeroGR is composed of three key components: (i) an LM-based docid generator that unifies heterogeneous documents (e.g., text, tables, code) into semantically meaningful docids; (ii) an instruction-tuned query generator that generates diverse types of queries from natural language task descriptions to enhance corpus indexing; and (iii) a reverse annealing decoding strategy to balance precision and recall during docid generation. Furthermore, we introduce OpenInstIR, the most diverse open-source instructed retrieval dataset. We investigate the impact of instruction fine-tuning scale and find that performance consistently improves as the number of IR tasks encountered during training increases. Extensive experiments on the BEIR and MAIR benchmarks demonstrate that ZeroGR achieves competitive performance across a wide range of retrieval tasks, establishing a new state-of-the-art among GR methods. Our code is available at https://github.com/sunnweiwei/ZeroGR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。