让AI在证据充分时才回答,避免误答。
Learning Evidence Sufficiency Boundaries for Selective Answering in Grounded Multi-Hop QA

- 构建有序证据链,直接监督模型从拒绝到回答的转变点。
- 在多个数据集上,拒答转回答的准确率达0.807,误答率降至0.095。
- 适合需要高可信度问答的应用,如医疗、法律等严谨场景。
基于证据的问答系统应在证据支持时才作答。在多跳问答中,部分证据可能使不支持的答案显得合理,导致判断困难。本文提出通过证据充分性边界来实现选择性回答:对同一问题,模型应在无证据或部分证据时拒绝回答,在证据首次充分时开始回答,并在添加冗余证据后保持答案稳定。我们引入证据充分性边界训练(Evidence Sufficiency Boundary Training),一种生成原生的训练框架,通过构建有序证据链并直接监督拒绝到回答的转换。该方法结合层级监督、边界翻转阈值、边界后稳定性与答案召回保护。我们在HotpotQA、2WikiMultiHopQA和MuSiQue上构建证据链,使用Qwen2.5-3B-Instruct与LoRA微调,评估指标包括链式推理得分、原始问答性能及外部非可答集上的未支持回答率。结果表明,该方法在边界定位上表现最优,翻转准确率达0.807(基线0.781),外部未支持回答率最低为0.095(基线0.101),同时保持了有竞争力的原始问答F1分数。说明在训练中明确标记拒绝转为回答的证据层级,能显著提升可信问答能力。
原文摘要 · Abstract (English)
Grounded question answering systems should answer only when the supplied evidence supports the answer. In multi-hop QA, this requirement is difficult because partial evidence can make an unsupported answer appear plausible. We study selective answering through evidence sufficiency boundaries: for the same question, a model should abstain under unsupported or partially supported context, answer when the context first becomes sufficient, and keep the answer stable when redundant evidence is added. We introduce Evidence Sufficiency Boundary Training, a generation-native training framework that constructs ordered evidence chains and supervises the abstain-to-answer transition directly. The method combines level supervision, a boundary flip margin, post-boundary stability, and answer recall protection. We build evidence chains from HotpotQA, 2WikiMultiHopQA, and MuSiQue, then evaluate models with chain metrics, raw QA utility, and unsupported-answer rates on external non-answerable sets. With Qwen2.5-3B-Instruct and LoRA adaptation, Evidence Sufficiency Boundary Training gives the strongest boundary localization among the tested systems, with flip accuracy of 0.807 compared with 0.781 for a token-level abstention baseline. It also achieves the lowest overall unsupported-answer rate on external non-answerable evaluation, 0.095 compared with 0.101 for the same baseline, while retaining competitive raw QA F1. The results show that grounded selective answering improves when training marks the evidence level where refusal should give way to answering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。