用少量标注样本增强生成逻辑，让大模型更讲真话。
Retrieval-augmented Decoding for Improving Truthfulness in Open-ended Generation
- 用10个标注样例构建语义检索库，动态调整生成词概率。
- 在4个模型、4个任务上均超越基线，事实准确率显著提升。
- 无需重训练，适合需要高可信度生成的场景。
确保大语言模型生成内容的真实性仍是可靠文本生成的关键挑战。尽管监督微调和基于人类反馈的强化学习表现良好，但需大量标注数据与计算资源，限制了可扩展性。相比之下，解码阶段干预无需模型重训练，更具轻量化优势。然而现有方法常面临提示敏感、泛化能力弱或依赖模型内部状态等问题。我们提出检索增强解码（RAD），一种上下文感知的自适应解码方法：仅需10个标注示例构建紧凑的参考锚定空间，该空间包含上下文嵌入与真实回答对应的下一词概率对，用于推理时的检索式概率调整。每个解码步骤中，RAD从锚定空间检索语义相似的上下文，并聚合其对应下一词概率以修正当前模型输出。在四个开放式生成基准与四个大模型上，本方法持续优于强基线，展现出稳健的跨任务泛化能力，凸显上下文感知解码在提升事实可靠性方面的潜力。
原文摘要 · Abstract (English)
Ensuring truthfulness in large language models (LLMs) remains a critical challenge for reliable text generation. While supervised fine-tuning and reinforcement learning with human feedback have shown promise, they require a substantial amount of annotated data and computational resources, limiting scalability. In contrast, decoding-time interventions offer lightweight alternatives without model retraining. However, existing decoding strategies often face issues like prompt sensitivity, limited generalization, or dependence on internal model states. We propose Retrieval-Augmented Decoding (RAD), a context-aware adaptive decoding method that leverages a compact reference grounding space built from as few as 10 annotated examples and comprising pairs of context embeddings and next-token logits from truthful responses, to enable retrieval-based logit shaping during inference. At each decoding step, RAD retrieves high-quality semantically similar contexts from this grounding space and aggregates their associated next token logits to modify the model's current logits. Across four open-ended generation benchmarks and four LLMs, our method consistently outperforms strong baselines and shows robust cross-task generalization, underscoring the promise of context-aware decoding for enhancing factual reliability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。