为检索增强问答提供不确定性量化方法，判断检索片段是否真正有用。
Uncertainty Quantification in Retrieval Augmented Question Answering
- 用轻量神经模型预测检索片段对答案的贡献度
- 比传统信息论指标更准，且比采样法更高效
- 适合关注问答可信度和可解释性的研究者
检索增强问答（RAG）通过在推理时引入检索到的文本片段，帮助问答模型弥补知识缺口。以往研究显示该方法能提升性能并减少幻觉，但未评估检索片段是否真有助于正确回答。本文提出通过估计片段实用性来量化问答模型的不确定性。训练一个轻量级神经模型预测目标QA模型所用片段的效用，结果表明：虽简单信息论指标可部分预测答案正确性，但本方法在效率上接近或超越更昂贵的采样策略。代码与数据已公开于 https://github.com/lauhaide/ragu。
原文摘要 · Abstract (English)
Retrieval augmented Question Answering (QA) helps QA models overcome knowledge gaps by incorporating retrieved evidence, typically a set of passages, alongside the question at test time. Previous studies show that this approach improves QA performance and reduces hallucinations, without, however, assessing whether the retrieved passages are indeed useful at answering correctly. In this work, we propose to quantify the uncertainty of a QA model via estimating the utility of the passages it is provided with. We train a lightweight neural model to predict passage utility for a target QA model and show that while simple information theoretic metrics can predict answer correctness up to a certain extent, our approach efficiently approximates or outperforms more expensive sampling-based methods. Code and data are available at https://github.com/lauhaide/ragu.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。