通过推理蒸馏与决策代理减少大模型检索幻觉，提升支付宝搜索准确性
Alleviating LLM-based Generative Retrieval Hallucination in Alipay Search
- 用大模型评估并蒸馏检索结果的推理逻辑，指导模型学习真实相关性
- 引入决策代理多角度筛选，使最终结果相关性显著提升
- 在支付宝基金与保险搜索中验证，线上转化率明显提高
生成式检索（GR）随大语言模型（LLM）的发展而革新，正被业界逐步采用。然而，基于LLM的GR常出现幻觉，在某些情况下生成与查询无关的文档，严重削弱其实际应用可信度。为此，我们提出一种优化的GR框架，以缓解检索幻觉问题，该框架在模型训练中融入知识蒸馏推理，并引入决策代理进一步提升检索精度。具体而言，我们利用LLM对生成的查询-文档（q-d）对进行评估和推理，并将推理数据作为转移知识蒸馏至GR模型。此外，我们使用决策代理作为后处理模块，结合检索模型扩展结果，并从多角度选择最相关的文档作为最终生成式检索结果。在真实世界数据集上的大量离线实验，以及在支付宝基金搜索和保险搜索中的在线A/B测试均表明，该框架在提升搜索质量与转化率方面具有显著优势。
原文摘要 · Abstract (English)
Generative retrieval (GR) has revolutionized document retrieval with the advent of large language models (LLMs), and LLM-based GR is gradually being adopted by the industry. Despite its remarkable advantages and potential, LLM-based GR suffers from hallucination and generates documents that are irrelevant to the query in some instances, severely challenging its credibility in practical applications. We thereby propose an optimized GR framework designed to alleviate retrieval hallucination, which integrates knowledge distillation reasoning in model training and incorporate decision agent to further improve retrieval precision. Specifically, we employ LLMs to assess and reason GR retrieved query-document (q-d) pairs, and then distill the reasoning data as transferred knowledge to the GR model. Moreover, we utilize a decision agent as post-processing to extend the GR retrieved documents through retrieval model and select the most relevant ones from multi perspectives as the final generative retrieval result. Extensive offline experiments on real-world datasets and online A/B tests on Fund Search and Insurance Search in Alipay demonstrate our framework's superiority and effectiveness in improving search quality and conversion gains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。