让大模型在生成答案前自动验证疑问句的多种解读是否合理。
Agentic Verification for Ambiguous Query Disambiguation
- 将多样化与验证结合，早期融合检索器和生成器反馈
- 在ASQA数据集上使答案相关性指标提升23%
- 适合需要高准确率的企业级问答系统
本文针对检索增强生成（RAG）中查询歧义消解问题，提出一种统一多样化与验证的新方法。现有方法采用先生成多样解读再验证的流程，易引入生成或检索噪声，尤其在企业场景下因模型训练数据静态而难以处理领域特异性歧义。为此，我们提出验证性多样化整合（VERDICT）方法，通过早期融合检索器与生成器反馈，减少对多次检索与推理步骤的依赖，从而提升效率与鲁棒性。在广泛使用的ASQA基准上验证，VERDICT在不同基础大模型下平均将接地感知F1分数提升23%，实现既多样又可验证的解读。
原文摘要 · Abstract (English)
In this work, we tackle the challenge of disambiguating queries in retrieval-augmented generation (RAG) to diverse yet answerable interpretations. State-of-the-arts follow a Diversify-then-Verify (DtV) pipeline, where diverse interpretations are generated by an LLM, later used as search queries to retrieve supporting passages. Such a process may introduce noise in either interpretations or retrieval, particularly in enterprise settings, where LLMs -- trained on static data -- may struggle with domain-specific disambiguations. Thus, a post-hoc verification phase is introduced to prune noises. Our distinction is to unify diversification with verification by incorporating feedback from retriever and generator early on. This joint approach improves both efficiency and robustness by reducing reliance on multiple retrieval and inference steps, which are susceptible to cascading errors. We validate the efficiency and effectiveness of our method, Verified-Diversification with Consolidation (VERDICT), on the widely adopted ASQA benchmark to achieve diverse yet verifiable interpretations. Empirical results show that VERDICT improves grounding-aware F1 score by an average of 23% over the strongest baseline across different backbone LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。