提出自适应解码方法CoCoA，提升大模型生成时对知识冲突的处理能力。
CoCoA: Confidence and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in Large Language Models
- 基于置信度与上下文特征动态调整解码策略
- 问答任务准确率提升最高达9.2点，摘要事实性提高2.5点
- 适合需要高忠实度生成的场景，如智能客服、内容创作
大语言模型在生成时面临参数记忆与外部上下文之间的知识冲突问题。现有对比解码方法虽能处理冲突，但缺乏自适应能力，在低冲突场景下性能下降。本文提出CoCoA（Confidence- and Context-Aware Adaptive Decoding），一种新的逐标记解码算法，通过置信度感知指标（熵差与上下文集中度）及参数分布与上下文分布间的广义散度，实现更合理的冲突化解。该方法在多种大模型上，于多个问答（QA）、摘要和长文本问答（LFQA）基准测试中表现优异，相比强基线AdaCAD平均提升9.2点问答准确率，摘要与LFQA的事实性平均提升2.5点。同时对冲突变化敏感，保持低冲突场景下的高性能。显著增强生成内容的准确性与上下文相关性。
原文摘要 · Abstract (English)
Faithful generation in large language models (LLMs) is challenged by knowledge conflicts between parametric memory and external context. Existing contrastive decoding methods tuned specifically to handle conflict often lack adaptability and can degrade performance in low conflict settings. We introduce CoCoA (Confidence- and Context-Aware Adaptive Decoding), a novel token-level algorithm for principled conflict resolution and enhanced faithfulness. CoCoA resolves conflict by utilizing confidence-aware measures (entropy gap and contextual peakedness) and the generalized divergence between the parametric and contextual distributions. Crucially, CoCoA maintains strong performance even in low conflict settings. Extensive experiments across multiple LLMs on diverse Question Answering (QA), Summarization, and Long-Form Question Answering (LFQA) benchmarks demonstrate CoCoA's state-of-the-art performance over strong baselines like AdaCAD. It yields significant gains in QA accuracy, up to 9.2 points on average compared to the strong baseline AdaCAD, and improves factuality in summarization and LFQA by up to 2.5 points on average across key benchmarks. Additionally, it demonstrates superior sensitivity to conflict variations. CoCoA enables more informed, context-aware, and ultimately more faithful token generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。