让大模型学会自我核查事实，减少幻觉同时不牺牲回答率。
Do I Really Know? Learning Factual Self-Verification for Hallucination Reduction
- 训练时引入自验证流程，让模型判断自己回答是否一致。
- 在多个模型上减少幻觉9.7%至53.3%，召回率下降不足6%。
- 适合需要高可信度问答的场景，如医疗、法律应用。
事实幻觉仍是大语言模型的核心挑战。现有方法多依赖外部后验验证或在微调中将不确定性直接映射为拒绝回答，常导致过度保守。我们提出VeriFY，一种训练时框架，通过一致性自验证教会模型推理事实不确定性。VeriFY在训练中引入结构化验证轨迹：模型先生成答案，再生成并回答一个探查性验证问题，做出一致性判断，最后决定是否回答或放弃。为防止训练中强化幻觉内容，我们设计阶段级损失掩码，排除幻觉回答阶段的训练目标，仅保留对验证行为的监督。在多个模型家族和规模下，VeriFY将事实幻觉率降低9.7%至53.3%，召回率仅下降0.4%至5.7%，且在单源数据训练后可在多数据集上泛化。代码、训练数据与模型检查点将在接受后公开。
原文摘要 · Abstract (English)
Factual hallucination remains a central challenge for large language models (LLMs). Existing mitigation approaches primarily rely on either external post-hoc verification or mapping uncertainty directly to abstention during fine-tuning, often resulting in overly conservative behavior. We propose VeriFY, a training-time framework that teaches LLMs to reason about factual uncertainty through consistency-based self-verification. VeriFY augments training with structured verification traces that guide the model to produce an initial answer, generate and answer a probing verification query, issue a consistency judgment, and then decide whether to answer or abstain. To address the risk of reinforcing hallucinated content when training on augmented traces, we introduce a stage-level loss masking approach that excludes hallucinated answer stages from the training objective while preserving supervision over verification behavior. Across multiple model families and scales, VeriFY reduces factual hallucination rates by 9.7 to 53.3 percent, with only modest reductions in recall (0.4 to 5.7 percent), and generalizes across datasets when trained on a single source. The source code, training data, and trained model checkpoints will be released upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。