通过自适应路径对比提升大模型生成可靠性，减少幻觉。
APCD: Adaptive Path-Contrastive Decoding for Reliable Large Language Model Generation
- 根据预测熵决定何时分叉，避免过早探索
- 路径间差异越大，相互影响越弱，保持多样性
- 在8个基准上提升事实准确性，适合追求可靠生成的场景
大语言模型在自回归解码过程中常因早期错误选择导致幻觉累积。虽然多路径解码可通过探索不同生成轨迹提升鲁棒性，但现有方法缺乏合理的分叉时机判断和路径间交互控制策略。本文提出自适应路径对比解码（APCD），通过两个组件实现：(1) 基于熵的路径扩展——当顶候选词的香农熵显示存在多个合理延续时才触发分叉；(2) 差异感知路径对比——鼓励多样化推理路径，并在各路径预测分布差异增大时动态减弱其相互影响。在八个基准测试上的实验表明，该方法在保持解码效率的同时显著提升了事实准确性。代码已开源：https://github.com/zty-king/APCD。
原文摘要 · Abstract (English)
Large language models (LLMs) often suffer from hallucinations due to error accumulation in autoregressive decoding, where suboptimal early token choices misguide subsequent generation. Although multi-path decoding can improve robustness by exploring alternative trajectories, existing methods lack principled strategies for determining when to branch and how to regulate inter-path interactions. We propose Adaptive Path-Contrastive Decoding (APCD), a multi-path decoding framework that improves output reliability through adaptive exploration and controlled path interaction. APCD consists of two components: (1) Entropy-Driven Path Expansion, which delays branching until predictive uncertainty - measured by Shannon entropy over top candidate tokens - indicates multiple plausible continuations; and (2) Divergence-Aware Path Contrast, which encourages diverse reasoning trajectories while dynamically attenuating inter-path influence as prediction distributions diverge. Experiments on eight benchmarks demonstrate improved factual accuracy while maintaining decoding efficiency. Our code is available at https://github.com/zty-king/APCD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。