让AI回答错误时自动找反例修正答案，提升事实问答准确率。
CounterRefine: Answer-Conditioned Counterevidence Retrieval for Inference-Time Knowledge Repair in Factual Question Answering
- 根据初答生成针对性反例查询，主动寻找可反驳证据。
- 在简单问答基准上比基线提升5.8个百分点正确率。
- 轻量修复层适合集成到现有系统，适合需要高精度的场景。
在事实性问答中，许多错误并非因无法获取证据，而是因错误坚持了答案。我们提出CounterRefine，一种面向短文本RAG的轻量级修复模块，将初始答案视为待检验假设。给定草稿答案后，CounterRefine生成与答案相关的扩展查询，检索候选特定证据，并通过受控的KEEP或REVISE步骤进行修正，新提议仅在确定性验证通过后才被采纳。该设计刻意保持简洁：仅增加一次证据检索和一次受控修正调用，而非替换检索器或构建复杂代理系统。在完整SimpleQA基准上，CounterRefine相比单次检索基线最多提升5.8个百分点正确率；在完整Claude追踪数据中，仅改变5.6%的输出，其中180次为有益修正，8次为有害变化。结果表明，具备知识的基础模型除了获取证据外，还应能利用证据重新审视并修复自身答案。
原文摘要 · Abstract (English)
In factual question answering, many errors are not failures of access but failures of commitment: the system retrieves relevant evidence, yet still settles on the wrong answer. We present CounterRefine, a lightweight repair layer for short-form RAG that treats the first answer as a hypothesis to test. Given a draft, CounterRefine issues answer-conditioned expansion queries to retrieve candidate-specific evidence, then applies a constrained KEEP or REVISE refinement step whose proposed revisions are accepted only after deterministic validation. The design is intentionally narrow: it adds one evidence-gathering pass and one guarded refinement call rather than replacing the retriever or building a broad agentic system. On the full SimpleQA benchmark, CounterRefine improves a matched one-pass RAG baseline by up to 5.8 correct-rate points; in the full Claude trace, it changes only 5.6% of outputs, with 180 beneficial outcome changes and 8 harmful ones. These findings suggest a simple but important direction for knowledgeable foundation models: beyond accessing evidence, they should also be able to use that evidence to reconsider and, when necessary, repair their own answers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。