arXiv:2506.11603cs.IR2025-06被引 13

小模型通过强化学习实现复杂查询推理,效果媲美大模型且部署更便宜。

TongSearch-QR: Reinforced Query Reasoning for Retrieval

  • 用半规则奖励函数指导小模型进行查询推理与重写。
  • 在BRIGHT基准上超越现有基线,7B和1.5B模型均表现优异。
  • 适合需要低成本、高适应性的实际检索系统使用。

传统信息检索方法在文本和语义匹配上表现良好,但在需要多跳推理或复杂语义理解的检索任务中表现不佳。一种有前景的解决方案是利用大语言模型(LLMs)显式重写或增强查询,以激发相关推理内容。然而,GPT-4或LLaMA3-70B等大规模模型因推理成本高,在真实系统中难以部署。本文提出TongSearch QR(原称TongSearch Reasoner),一套用于复杂检索任务中查询推理与重写的轻量级语言模型。通过新颖的半规则化奖励函数,结合强化学习,使Qwen2.5-7B-Instruct和Qwen2.5-1.5B-Instruct等小型模型达到媲美大模型的推理性能,同时显著降低计算开销。在BRIGHT基准上的实验表明,当搭配BM25检索器时,TongSearch QR-7B与TongSearch QR-1.5B均显著优于现有基线,包括基于提示的查询推理器及部分最新稠密检索器,展现出更强的真实场景部署适应性。

原文摘要 · Abstract (English)

Traditional information retrieval (IR) methods excel at textual and semantic matching but struggle in reasoning-intensive retrieval tasks that require multi-hop inference or complex semantic understanding between queries and documents. One promising solution is to explicitly rewrite or augment queries using large language models (LLMs) to elicit reasoning-relevant content prior to retrieval. However, the widespread use of large-scale language models like GPT-4 or LLaMA3-70B remains impractical due to their high inference cost and limited deployability in real-world systems. In this work, we introduce TongSearch QR (Previously Known as "TongSearch Reasoner"), a family of small-scale language models for query reasoning and rewriting in reasoning-intensive retrieval. With a novel semi-rule-based reward function, we employ reinforcement learning approaches enabling smaller language models, e,g, Qwen2.5-7B-Instruct and Qwen2.5-1.5B-Instruct, to achieve query reasoning performance rivaling large-scale language models without their prohibitive inference costs. Experiment results on BRIGHT benchmark show that with BM25 as retrievers, both TongSearch QR-7B and TongSearch QR-1.5B models significantly outperform existing baselines, including prompt-based query reasoners and some latest dense retrievers trained for reasoning-intensive retrieval tasks, offering superior adaptability for real-world deployment.

查询重写强化学习小模型信息检索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。