预测检索是否提升问答质量，实现智能选择
Predicting the Benefit of Retrieval Augmentation in Open-Domain Question Answering
- 基于检索信号与生成一致性设计预测方法
- 动态选择是否使用检索，性能超越全量检索
- 适合希望提升效率的问答系统开发者
尽管检索增强生成已成为提升开放域问答系统性能的常用方法，但检索并非对所有问题都有利。本文研究如何预测在给定问题上引入外部检索信息是否能改善回答质量。我们评估了多种基于检索信号、答案特征及生成回答与检索片段语义一致性的预测方法，并提出一种探测大模型内部状态的新型预测器。该预测器显著缩小了后生成方法（计算开销大）与前生成（检索后）方法之间的性能差距。利用这些预测方法，我们构建了一个选择性检索框架，根据每个问题动态决定采用检索或非检索生成模式。实验表明，有选择地应用检索增强，可获得优于对所有问题均使用检索的系统性能。
原文摘要 · Abstract (English)
While retrieval augmented generation has become a common approach for enhancing question answering systems, retrieval is not universally advantageous. We study the problem of predicting whether incorporating external retrieved information is likely to improve response quality for a given question. To this end, we evaluate a range of prediction methods that are based on retrieval signals, answer characteristics, and semantic consistency between generated responses and retrieved passages. We further devise a predictor that probes the LLM's internal state. Its prediction performance significantly narrows the performance gap between post-generation methods which are computationally demanding and pre-generation (post-retrieval) methods. We use the prediction methods to devise a selective retrieval framework that dynamically chooses between retrieval and non-retrieval generation modes per question. Experimental results demonstrate that selectively applying retrieval augmentation yields answer quality that transcends that of using retrieval for all queries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。