arXiv:2505.21439cs.CLcs.IR2025-05被引 8

构建指令跟随检索数据集,让模型更懂用户意图。

Towards Better Instruction Following Retrieval Models

  • 用3.8万组指令-查询-段落三元组训练模型,提升理解力。
  • 新模型InF-Embed在5个基准上指令遵循能力提升8.1%。
  • 适合需要精准理解用户指令的搜索系统开发者。

当前信息检索模型仅基于标准的<查询, 段落>对训练,难以有效理解并执行用户明确指令。本文提出InF-IR,一个大规模高质量的指令跟随检索训练语料库,将传统训练样本扩展为超过38,000个富有表现力的<指令, 查询, 段落>三元组作为正样本。针对每个正三元组,通过污染指令和查询生成两个额外的困难负例,并由先进的推理模型(o3-mini)严格验证,确保语义合理但指令错误。与现有主要支持解码器模型重排序任务的语料库不同,InF-IR中的高对比度正负三元组可有效支持更小的编码器模型进行高效表示学习,实现直接嵌入式检索。基于该语料库,我们训练了InF-Embed,一种通过对比学习和指令-查询注意力机制优化的指令感知嵌入模型,使检索结果精确对齐用户意图。在五个基于指令的检索基准上的大量实验表明,InF-Embed相比竞争基线在p-MRR指标上显著提升8.1%,衡量其指令遵循能力。

原文摘要 · Abstract (English)

Modern information retrieval (IR) models, trained exclusively on standard <query, passage> pairs, struggle to effectively interpret and follow explicit user instructions. We introduce InF-IR, a large-scale, high-quality training corpus tailored for enhancing retrieval models in Instruction-Following IR. InF-IR expands traditional training pairs into over 38,000 expressive <instruction, query, passage> triplets as positive samples. In particular, for each positive triplet, we generate two additional hard negative examples by poisoning both instructions and queries, then rigorously validated by an advanced reasoning model (o3-mini) to ensure semantic plausibility while maintaining instructional incorrectness. Unlike existing corpora that primarily support computationally intensive reranking tasks for decoder-only language models, the highly contrastive positive-negative triplets in InF-IR further enable efficient representation learning for smaller encoder-only models, facilitating direct embedding-based retrieval. Using this corpus, we train InF-Embed, an instruction-aware Embedding model optimized through contrastive learning and instruction-query attention mechanisms to align retrieval outcomes precisely with user intents. Extensive experiments across five instruction-based retrieval benchmarks demonstrate that InF-Embed significantly surpasses competitive baselines by 8.1% in p-MRR, measuring the instruction-following capabilities.

信息检索指令跟随嵌入模型对比学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。