研究大模型如何权衡自身知识与用户、文档信息，提升可信度。
How Large Language Models Balance Internal Knowledge with User and Document Assertions
- 构建三源交互框架，同时考察模型、用户和文档的相互影响。
- 多数模型更信文档而非用户，且训练后倾向更强。
- 微调可显著提升模型辨别有害信息能力，适合安全应用研发者。
大语言模型在真实场景中常需平衡内部参数化知识与外部信息（如用户信念或检索文档）。模型处理这些信息的能力直接影响系统安全性。以往研究局限于模型知识与文档或用户间的二元冲突，忽略了三者共存的复杂环境。为此，我们提出三源交互框架，系统评估了3个家族共27个LLM在2个数据集上的表现。结果表明：大多数模型更依赖文档陈述，且该倾向在后训练阶段被强化；行为分析显示，多数模型易受外部信息影响，难以区分有益与有害内容。通过在多样化源交互数据上微调，可显著提升模型的辨别能力。本工作为构建可信赖的多源信息融合模型提供了路径。代码已开源。
原文摘要 · Abstract (English)
Large language models (LLMs) often need to balance their internal parametric knowledge with external information, such as user beliefs and content from retrieved documents, in real-world scenarios like RAG or chat-based systems. A model's ability to reliably process these sources is key to system safety. Previous studies on knowledge conflict and sycophancy are limited to a binary conflict paradigm, primarily exploring conflicts between parametric knowledge and either a document or a user, but ignoring the interactive environment where all three sources exist simultaneously. To fill this gap, we propose a three-source interaction framework and systematically evaluate 27 LLMs from 3 families on 2 datasets. Our findings reveal general patterns: most models rely more on document assertions than user assertions, and this preference is reinforced by post-training. Furthermore, our behavioral analysis shows that most models are impressionable, unable to effectively discriminate between helpful and harmful external information. To address this, we demonstrate that fine-tuning on diverse source interaction data can significantly increase a model's discrimination abilities. In short, our work paves the way for developing trustworthy LLMs that can effectively and reliably integrate multiple sources of information. Code is available at https://github.com/shuowl/llm-source-balancing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。