研究大模型如何在问答中平衡上下文与参数知识，发现70%依赖上下文。
When Context Leads but Parametric Memory Follows in Large Language Models
- 通过控制上下文大小，分析模型对上下文与参数知识的分配比例。
- 约70%知识来自上下文，30%来自参数，且上下文越多，幻觉越少。
- 适合关注模型推理机制与减少幻觉的研究者参考。
大型语言模型（LLMs）在利用多样知识源方面取得了显著进展。本研究调查了九种广泛使用的LLM在知识一致场景下回答开放性问题时，如何在局部上下文和全局参数之间分配知识。我们引入了一个新数据集WikiAtomic，系统性地改变上下文大小，分析模型在知识一致场景中对提供信息与参数化知识的优先级和利用方式。此外，还研究了不同上下文大小下模型产生幻觉的倾向。研究发现，各类模型均呈现一致模式：对上下文知识的依赖度约为70%，对参数知识的依赖度约为30%，且随着上下文增大，幻觉现象明显减少。这些发现强调了更有效组织上下文以及开发能更确定性使用输入的模型的重要性，以实现稳健性能。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated remarkable progress in leveraging diverse knowledge sources. This study investigates how nine widely used LLMs allocate knowledge between local context and global parameters when answering open-ended questions in knowledge-consistent scenarios. We introduce a novel dataset, WikiAtomic, and systematically vary context sizes to analyze how LLMs prioritize and utilize the provided information and their parametric knowledge in knowledge-consistent scenarios. Additionally, we also study their tendency to hallucinate under varying context sizes. Our findings reveal consistent patterns across models, including a consistent reliance on both contextual (around 70%) and parametric (around 30%) knowledge, and a decrease in hallucinations with increasing context. These insights highlight the importance of more effective context organization and developing models that use input more deterministically for robust performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。