arXiv:2409.16146cs.CL2024-09EMNLP被引 17

提出反事实提示框架,让RAG模型主动拒绝低信度回答。

Controlling Risk of Retrieval-augmented Generation: A Counterfactual Prompting Framework

  • 通过反事实提示诱导模型反思检索质量与使用方式对信心的影响。
  • 在多个数据集上实现90%以上高风险场景下的拒答率,同时保持85%以上准确率。
  • 适合需要高可靠性输出的医疗、金融等关键领域应用。

检索增强生成(RAG)已成为缓解大语言模型幻觉问题的流行方案。然而,现有研究极少关注预测不确定性——即RAG模型预测错误的可能性——导致实际应用中风险不可控。本文强调风险控制的重要性,确保RAG模型能主动拒绝低置信度问题的回答。研究识别出影响RAG置信度的两个关键潜在因素:检索结果的质量和结果利用方式。为此,我们提出一种反事实提示框架,引导模型改变这两个因素并分析其对答案的影响。同时引入可选择拒答的基准测试流程,支持系列实验。评估中设计多种风险相关指标,实验结果证明该方法有效。代码与基准数据集已开源。

原文摘要 · Abstract (English)

Retrieval-augmented generation (RAG) has emerged as a popular solution to mitigate the hallucination issues of large language models. However, existing studies on RAG seldom address the issue of predictive uncertainty, i.e., how likely it is that a RAG model's prediction is incorrect, resulting in uncontrollable risks in real-world applications. In this work, we emphasize the importance of risk control, ensuring that RAG models proactively refuse to answer questions with low confidence. Our research identifies two critical latent factors affecting RAG's confidence in its predictions: the quality of the retrieved results and the manner in which these results are utilized. To guide RAG models in assessing their own confidence based on these two latent factors, we develop a counterfactual prompting framework that induces the models to alter these factors and analyzes the effect on their answers. We also introduce a benchmarking procedure to collect answers with the option to abstain, facilitating a series of experiments. For evaluation, we introduce several risk-related metrics and the experimental results demonstrate the effectiveness of our approach. Our code and benchmark dataset are available at https://github.com/ict-bigdatalab/RC-RAG.

RAG风险控制提示工程可信AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。