让大模型生成时实时自检纠错，提升事实准确性
DSVD: Dynamic Self-Verify Decoding for Faithful Generation in Large Language Models
- 生成过程同步自检，发现幻觉立即修正
- 在五个基准上显著提升事实准确率，最高增12.3%
- 可无缝集成现有方法，适合追求高可靠性的应用
大语言模型的可靠性仍是关键挑战，尤其在生成文本时易产生幻觉和事实错误。现有方法或未能充分利用模型的自我修正能力，或采用代价高昂的事后验证。为探索生成过程中的实时自验证与纠错潜力，我们提出动态自验证解码（DSVD），通过实时检测幻觉并高效修复错误来增强生成可靠性。DSVD包含两个核心组件：(1) 并行自验证架构，实现持续质量评估；(2) 动态回滚机制，进行精准错误恢复。在五个基准上的大量实验表明，DSVD在真实性（Question-Answering）和事实准确性（FActScore）方面均取得显著提升。结果还显示，DSVD可进一步与现有可信解码方法结合，获得更强性能。本工作证实，生成过程中实时自验证是实现更可信语言模型的可行路径，且不牺牲实际部署可行性。
原文摘要 · Abstract (English)
The reliability of large language models remains a critical challenge, particularly due to their susceptibility to hallucinations and factual inaccuracies during text generation. Existing solutions either underutilize models' self-correction with preemptive strategies or use costly post-hoc verification. To further explore the potential of real-time self-verification and correction, we present Dynamic Self-Verify Decoding (DSVD), a novel decoding framework that enhances generation reliability through real-time hallucination detection and efficient error correction. DSVD integrates two key components: (1) parallel self-verification architecture for continuous quality assessment, (2) dynamic rollback mechanism for targeted error recovery. Extensive experiments across five benchmarks demonstrate DSVD's effectiveness, achieving significant improvement in truthfulness (Quesetion-Answering) and factual accuracy (FActScore). Results show the DSVD can be further incorporated with existing faithful decoding methods to achieve stronger performance. Our work establishes that real-time self-verification during generation offers a viable path toward more trustworthy language models without sacrificing practical deployability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。