现有视觉语言模型无法正确理解物体物理变化的守恒性。
Vision Language Models Cannot Reason About Physical Transformation
- 构建ConservationBench评测框架，测试物理量在变换下的不变性。
- 23,040个问题中模型表现接近随机，守恒任务提升伴随对照组下降。
- 模型依赖文字先验，实际视觉信息处理能力差，适合研究者关注物理推理短板。
理解物理变换是动态环境中推理的基础。尽管视觉语言模型（VLMs）在具身应用中展现出潜力，但其是否真正理解物理变换仍不明确。我们提出ConservationBench，用于评估物理量在变换下是否保持守恒——涵盖四种属性，设计成配对的守恒/非守恒场景，生成并评估了23,040个问题，覆盖112种VLM。结果揭示系统性失败：模型性能接近随机，且守恒任务上的提升伴随着对照组表现下降。控制实验表明，模型受强烈文本先验驱动，倾向于假设不变性；但在真实视觉内容下，当守恒与非守恒场景平衡时，模型表现反而更差。时间分辨率、提示工程或精心采样均无法改善。这些发现表明，当前VLM无法在动态场景中维持物理属性的变换不变表示。
原文摘要 · Abstract (English)
Understanding physical transformations is fundamental for reasoning in dynamic environments. While Vision Language Models (VLMs) show promise in embodied applications, whether they genuinely understand physical transformations remains unclear. We introduce ConservationBench evaluating conservation -- whether physical quantities remain invariant under transformations. Spanning four properties with paired conserving/non-conserving scenarios, we generate and evaluate 23,040 questions across 112 VLMs. Results reveal systematic failure: performance remains near chance with improvements on conservation tasks accompanied by drops on controls. Control experiments show strong textual priors favoring invariance, yet models perform worse with actual visual content when performance is balanced across conserving and non-conserving scenarios. Neither temporal resolution, prompting, nor curated sampling helps. These findings show that current VLMs fail to maintain transformation-invariant representations of physical properties across dynamic scenes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。