增加上下文示例能降低大模型生成不确定性，提升可信度。
Uncertainty Unveiled: Can Exposure to More In-context Examples Mitigate Uncertainty for Large Language Models?
- 通过分解不确定性，发现额外示例主要减少认知不确定性
- 复杂任务中，示例增多后总不确定性下降，但需克服长输入噪声
- 揭示了模型内部置信度随层演进的机制，解释为何更优
近期长序列处理技术的进步推动了长上下文在上下文学习（ICL）中的应用。尽管现有研究多关注增加示例带来的性能提升，但其对生成结果可信度的影响仍不明确。本文系统量化了不同示例数量下ICL的预测不确定性，重点分析任务示例数量的影响。通过不确定性分解，提出新视角：性能提升主要源于减少认知不确定性（EU）。结果显示，无论简单或复杂任务，增加示例均通过注入任务特定知识降低总不确定性，从而改善表现。复杂任务中，该优势仅在克服长输入带来的噪声与不确定性后显现。此外，我们分析了模型各层内部置信度的演变，揭示了不确定性下降的内在机制。
原文摘要 · Abstract (English)
Recent advances in handling long sequences have facilitated the exploration of long-context in-context learning (ICL). While much of the existing research emphasizes performance improvements driven by additional in-context examples, the influence on the trustworthiness of generated responses remains underexplored. This paper addresses this gap by investigating how increased examples influence predictive uncertainty, an essential aspect in trustworthiness. We begin by systematically quantifying the uncertainty of ICL with varying shot counts, analyzing the impact of example quantity. Through uncertainty decomposition, we introduce a novel perspective on performance enhancement, with a focus on epistemic uncertainty (EU). Our results reveal that additional examples reduce total uncertainty in both simple and complex tasks by injecting task-specific knowledge, thereby diminishing EU and enhancing performance. For complex tasks, these advantages emerge only after addressing the increased noise and uncertainty associated with longer inputs. Finally, we explore the evolution of internal confidence across layers, unveiling the mechanisms driving the reduction in uncertainty.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。