专为推理任务训练的检索器，显著提升复杂问题的问答效果。
ReasonIR: Training Retrievers for Reasoning Tasks
- 用合成数据生成挑战性查询与干扰负例，优化训练
- 在BRIGHT基准上达29.9 nDCG@10（无重排序）和36.9（有重排序）
- 适合需要深度推理的RAG应用，且能更好利用大模型重排序
我们提出ReasonIR-8B，首个专门针对通用推理任务训练的检索器。现有检索器在推理任务中表现有限，部分原因在于训练数据集中短事实查询与直接答案文档绑定。为此，我们构建了合成数据生成流程：对每篇文档生成一个具有挑战性且相关的问题，以及一个看似相关但最终无用的硬负例。通过混合训练自建合成数据与公开数据，ReasonIR-8B在广泛使用的推理密集型信息检索基准BRIGHT上达到29.9 nDCG@10（无重排序）和36.9 nDCG@10（有重排序）的新纪录。应用于RAG任务时，相比闭卷基线，其在MMLU和GPQA上的性能分别提升6.4%和22.6%，优于其他检索器和搜索引擎。此外，ReasonIR-8B更高效利用测试阶段计算：在BRIGHT上，随着重写查询变长、信息量增加，性能持续提升；与LLM重排序器结合后仍保持领先。该训练方法具可扩展性，适用于未来大模型。代码、数据与模型已开源。
原文摘要 · Abstract (English)
We present ReasonIR-8B, the first retriever specifically trained for general reasoning tasks. Existing retrievers have shown limited gains on reasoning tasks, in part because existing training datasets focus on short factual queries tied to documents that straightforwardly answer them. We develop a synthetic data generation pipeline that, for each document, our pipeline creates a challenging and relevant query, along with a plausibly related but ultimately unhelpful hard negative. By training on a mixture of our synthetic data and existing public data, ReasonIR-8B achieves a new state-of-the-art of 29.9 nDCG@10 without reranker and 36.9 nDCG@10 with reranker on BRIGHT, a widely-used reasoning-intensive information retrieval (IR) benchmark. When applied to RAG tasks, ReasonIR-8B improves MMLU and GPQA performance by 6.4% and 22.6% respectively, relative to the closed-book baseline, outperforming other retrievers and search engines. In addition, ReasonIR-8B uses test-time compute more effectively: on BRIGHT, its performance consistently increases with longer and more information-rich rewritten queries; it continues to outperform other retrievers when combined with an LLM reranker. Our training recipe is general and can be easily extended to future LLMs; to this end, we open-source our code, data, and model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。