arXiv:2505.19509cs.LGcs.AI2025-05被引 17

构建多模态知识冲突评测基准,揭示大模型偏信自身知识的缺陷

Benchmarking Multimodal Knowledge Conflict for Large Multimodal Models

  • 提出三类真实场景下的多模态知识冲突评测框架
  • 覆盖1573个知识实例与3381张图像,支持内外知识冲突检测
  • 发现主流大模型更依赖内部知识而非外部证据

大型多模态模型(LMMs)在检索增强生成(RAG)框架下常面临多模态知识冲突,外部上下文信息可能与模型内嵌参数知识相矛盾,导致输出不可靠。现有基准未能充分反映此类真实冲突场景,多数仅关注内存内冲突,而对上下文-记忆和跨上下文冲突研究不足,且缺乏对事实性知识评估的重视。为此,我们提出MMKC-Bench,一个用于评估上下文-记忆与跨上下文场景中事实性知识冲突的基准。该基准涵盖三类多模态知识冲突,包含1,573个知识实例与3,381张图像,覆盖23个广泛类别,通过自动化管道收集并经人工验证。我们在三类代表性LMM上进行了模型行为分析与冲突检测任务评估。结果表明,尽管当前模型具备识别知识冲突的能力,但倾向于优先采纳内部参数化知识而非外部证据。我们希望此基准能推动多模态知识冲突研究,促进多模态RAG系统的发展。代码已开源。

原文摘要 · Abstract (English)

Large Multimodal Models(LMMs) face notable challenges when encountering multimodal knowledge conflicts, particularly under retrieval-augmented generation(RAG) frameworks where the contextual information from external sources may contradict the model's internal parametric knowledge, leading to unreliable outputs. However, existing benchmarks fail to reflect such realistic conflict scenarios. Most focus solely on intra-memory conflicts, while context-memory and inter-context conflicts remain largely investigated. Furthermore, commonly used factual knowledge-based evaluations are often overlooked, and existing datasets lack a thorough investigation into conflict detection capabilities. To bridge this gap, we propose MMKC-Bench, a benchmark designed to evaluate factual knowledge conflicts in both context-memory and inter-context scenarios. MMKC-Bench encompasses three types of multimodal knowledge conflicts and includes 1,573 knowledge instances and 3,381 images across 23 broad types, collected through automated pipelines with human verification. We evaluate three representative series of LMMs on both model behavior analysis and conflict detection tasks. Our findings show that while current LMMs are capable of recognizing knowledge conflicts, they tend to favor internal parametric knowledge over external evidence. We hope MMKC-Bench will foster further research in multimodal knowledge conflict and enhance the development of multimodal RAG systems. The source code is available at https://github.com/MLLMKCBENCH/MLLMKC.

多模态知识冲突RAG评测基准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。