测试大模型在知识冲突下的表现，发现其处理矛盾信息时效果下降。
Who's Who: Large Language Models Meet Knowledge Conflicts in Practice
- 构建了含5000个问题的基准数据集WhoQA，用于评估模型在知识冲突中的行为
- 在检索增强生成中，模型面对知识冲突时准确率显著下降
- 适合关注大模型可靠性与可解释性的研究人员使用
检索增强生成（RAG）方法是解决预训练语言模型静态记忆限制的有效方案。然而，在检索上下文中遇到相互冲突的信息源是实际应用中不可避免的挑战。此时，语言模型应透明地告知用户存在冲突，而非基于自身固有偏见自主决定呈现内容。为分析当前大语言模型（LLMs）是否符合此建议，我们提出WhoQA，一个公开的基准数据集，用于考察模型在知识冲突情境下的表现。通过询问同名实体的共同属性，生成最多包含8个不同答案的问题。WhoQA评估集涵盖13种Wikidata属性类型、150,000个维基百科实体，共5,000个问题。实验表明，尽管WhoQA问题形式简单，知识冲突仍显著降低模型在RAG设置中的表现。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) methods are viable solutions for addressing the static memory limits of pre-trained language models. Nevertheless, encountering conflicting sources of information within the retrieval context is an inevitable practical challenge. In such situations, the language models are recommended to transparently inform users about the conflicts rather than autonomously deciding what to present based on their inherent biases. To analyze how current large language models (LLMs) align with our recommendation, we introduce WhoQA, a public benchmark dataset to examine model's behavior in knowledge conflict situations. We induce conflicts by asking about a common property among entities having the same name, resulting in questions with up to 8 distinctive answers. WhoQA evaluation set includes 5K questions across 13 Wikidata property types and 150K Wikipedia entities. Our experiments show that despite the simplicity of WhoQA questions, knowledge conflicts significantly degrades LLMs' performance in RAG settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。