让大模型更懂复杂长查询,通过拆解与解释提升检索效果。
Reason to Retrieve: Enhancing Query Understanding through Decomposition and Interpretation
- 将复杂查询拆解为子问题,逐个理解用户意图。
- 在子查询中加入语义解释,显著提升文档匹配准确率。
- 适合需要精准理解长文本查询的搜索系统开发者。
查询理解(QU)旨在准确推断用户意图以改进文档检索,在现代搜索引擎中至关重要。尽管大语言模型(LLMs)在此领域取得进展,但其效果主要针对短关键词查询。随着人工智能驱动的搜索兴起,包含复杂意图的长查询日益普遍,但尚未被充分研究。为此,我们提出ReDI,一种通过分解与解释增强推理能力的查询理解方法。ReDI采用三阶段流程:(i) 将复杂查询分解为多个目标子查询以捕捉用户意图;(ii) 为每个子查询添加详细语义解释,增强意图-文档匹配;(iii) 独立检索各子查询结果后,使用融合策略整合输出最终排序。我们从商业搜索引擎收集大规模真实世界复杂查询数据集,并将DeepSeek-R1的查询理解能力蒸馏至小型模型以实现实用部署。在公开基准BRIGHT和BEIR上的实验表明,ReDI在稀疏与密集检索范式下均持续优于强基线,验证了其有效性。代码、生成的子查询与解释已开源:https://github.com/youngbeauty250/ReDI。
原文摘要 · Abstract (English)
Query understanding (QU) aims to accurately infer user intent to improve document retrieval. It plays a vital role in modern search engines. While large language models (LLMs) have made notable progress in this area, their effectiveness has primarily been studied on short, keyword-based queries. With the rise of AI-driven search, long-form queries with complex intent become increasingly common, but they are underexplored in the context of LLM-based QU. To address this gap, we introduce ReDI, a reasoning-enhanced query understanding method through decomposition and interpretation. ReDI uses the reasoning and understanding capabilities of LLMs within a three-stage pipeline. (i) It decomposes a complex query into a set of targeted sub-queries to capture the user intent. (ii) It enriches each sub-query with detailed semantic interpretations to enhance the retrieval of intent-document matching. And (iii), after independently retrieving documents for each sub-query, ReDI uses a fusion strategy to aggregate the results and obtain the final ranking. We collect a large-scale dataset of real-world complex queries from a commercial search engine and distill the query understanding capabilities of DeepSeek-R1 into small models for practical application. Experiments on public benchmarks, including BRIGHT and BEIR, show that ReDI consistently outperforms strong baselines in both sparse and dense retrieval paradigms, demonstrating its effectiveness. We release our code, generated sub-queries, and interpretations at https://github.com/youngbeauty250/ReDI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。