arXiv:2605.29339cs.CV2026-05

构建真实世界多模态因果反事实问答基准,评估模型真实因果推理能力

DMC-CF: Dynamic Multimodal CounterFactual QA benchmark for Causal Reasoning

论文配图:DMC-CF: Dynamic Multimodal CounterFactual QA benchmark for Causal Reasoning
图 1 · 摘自论文原文
  • 基于真实视频构建静态与动态双模式反事实评测集
  • 现有多模态大模型在真实场景下因果推理能力仍严重不足
  • 提出动态图干预框架,避免传统评估中的数据污染问题

随着多模态大语言模型(MLLMs)的快速发展,模型展现出日益强大的多模态能力。然而,通过统计学习训练的MLLM是否真正理解现实世界背后的因果关系,仍是关键研究问题。近年来虽涌现出多个多模态因果推理数据集,但普遍存在规模有限、内容依赖合成图像/视频、卡通化素材等非真实来源的问题。为此,本文收集真实世界视频,构建大规模多模态因果反事实推理基准DMC-CF-Static。为解决传统静态评估中的数据泄露问题,采用因果图表示因果事件,并提出动态图干预(DGI)框架,从静态基准构建动态评估基准DMC-CF-Dynamic。整体实验结果表明,当前多模态大模型在真实场景下的多模态因果推理能力仍有待显著提升。

原文摘要 · Abstract (English)

With the rapid advancement of multimodal large language models (MLLMs), models have demonstrated increasingly powerful multimodal capabilities. However, whether MLLMs trained through statistical learning can truly understand the causal relationships underlying the real world remains a key research question. In recent years, numerous multimodal causal reasoning datasets have been proposed. Nevertheless, these datasets are either limited in scale or constructed from synthetic images and videos, cartoon-based content, or other non-realistic multimodal sources. To address these limitations, we collect real-world videos and construct DMC-CF-Static, a large-scale benchmark for multimodal causal counterfactual reasoning. Furthermore, to mitigate issues such as data contamination in traditional static evaluation, we represent causal events using causal graphs and propose the Dynamic Graph Intervention (DGI) framework to build the dynamic evaluation benchmark DMC-CF-Dynamic from DMC-CF-Static. Experimental results on the overall DMC-CF, which includes both static and dynamic evaluation benchmarks, demonstrate that the multimodal causal reasoning capabilities of current multimodal large language models in real-world scenarios still require substantial improvement.

因果推理多模态评测基准反事实

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。