用检索到的查询变体提升神经排序模型的查询性能预测效果
RAQG-QPP: Query Performance Prediction with Retrieved Query Variants and Retrieval Augmented Query Generation
- 从历史查询日志中获取查询变体，结合大模型生成更相关的变体
- 在TREC DL'19和DL'20上，对MonoT5等神经模型提升达30%
- 适合需要无监督查询性能预测的搜索系统优化场景
查询性能预测（QPP）无需人工标注即可估计排序模型的检索质量，适用于查询特定的选择性决策以提升整体检索效果。尽管无监督QPP方法对词汇检索模型有效，但对神经排序模型表现较弱。近期研究发现利用查询变体（QVs）——即与给定查询信息需求可能相似的查询——可提升无监督QPP准确率。然而，现有基于查询变体的方法依赖输入查询的词项扩展生成变体，易产生不连贯、幻觉或无关的变体。本文提出使用过去查询日志中检索到的查询作为QVs，并进一步利用大语言模型（LLMs）基于这些检索到的变体生成新变体，缓解仅依赖日志查询的局限性。在TREC DL'19和DL'20上的实验表明，结合RAQG的QPP在如MonoT5等神经排序模型上，相比最优现有方法提升最高达30%。
原文摘要 · Abstract (English)
Query Performance Prediction (QPP) estimates the retrieval quality of ranking models without the use of any human-assessed relevance judgements, and finds applications in query-specific selective decision making to improve overall retrieval effectiveness. Although unsupervised QPP approaches are effective for lexical retrieval models, they usually perform weaker for neural rankers. Recent work shows that leveraging query variants (QVs), i.e., queries with potentially similar information needs to a given query, can enhance unsupervised QPP accuracy. However, existing QV-based prediction methods rely on query variants generated by term expansion of the input query, which is likely to yield incoherent, hallucinatory and off-topic QVs. In this paper, we propose to make use of queries retrieved from a log of past queries as QVs to be subsequently used for QPP. In addition to directly applying retrieved QVs in QPP, we further propose to leverage large language models (LLMs) to generate QVs conditioned on the retrieved QVs, thus mitigating the limitation of relying only on existing queries in a log. Experiments on TREC DL'19 and DL'20 show that QPP enhanced with RAQG outperform the best-performing existing QV-based prediction approach by as much as 30% on neural ranking models such as MonoT5.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。