让视觉语言模型学会内化因果推理，提升多图场景下的因果判断能力。
From Prompts to Tokens: Internalizing Causal Supervision in Vision-Language Model for Multi-Image Causal Reasoning
- 通过构建视觉因果图并转化为结构化因果令牌，实现因果推理的内部化执行。
- 在CausalVLBench上干预任务准确率达54.4%，较提示监督提升21.2个百分点。
- 适用于需要精准因果分析的多图像理解任务，如医疗影像、自动驾驶等场景。
视觉因果推理对于理解与干预物理世界至关重要，需从视觉输入中识别因果变量并推理干预效果。尽管已有进展，大型视觉-语言模型（VLM）在多图像输入的干预与反事实查询上仍表现脆弱。现有方法多通过文本提示注入因果知识，导致因果机制外置于模型运行过程，限制了推理时的可控性。为此，我们提出BridgeVLM，通过从多图像输入中推导因果图，并将其转换为由RAMP层注入LLM解码器的结构化因果令牌，实现因果消息传递的内部化执行。我们进一步设计统一训练接口M3S，支持不同粒度（局部/全局）的细粒度因果监督。BridgeVLM在CausalVLBench上的干预任务准确率达54.4%（对比提示监督的33.2%），将Causal3D性能从43.6%提升至49.0%，并在CausalVLBench上显著改进因果结构学习（F₁: 33.4% → 75.1%）。
原文摘要 · Abstract (English)
Visual causal reasoning is essential for understanding and intervening in the physical world, requiring identification of causal variables from visual inputs and reasoning over intervention effects. Despite recent progress, large vision--language models (VLMs) remain brittle at such tasks, especially for interventional and counterfactual queries over multi-image inputs. Most existing explorations inject causal knowledge via textual prompts, leaving causal mechanisms external to model execution and limiting reliable control during inference. To address this problem, we propose BridgeVLM, which internalizes visual causal reasoning by inducing a causal graph from multi-image inputs and converting it into structured Causal Tokens executed by RAMP layers injected into the LLM decoder for causal message passing. We further introduce a unified training interface M3S for fine-grained causal supervision from different granularities (local/global level). BridgeVLM achieves 54.4% accuracy on intervention tasks on CausalVLBench (vs. 33.2% with prompt-level supervision), improves results on Causal3D from 43.6% to 49.0%, and substantially improves causal structure learning on CausalVLBench ($F_1$: 33.4% $\rightarrow$ 75.1%).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。