用可执行代码评估大模型物理推理能力，让假设可检验、可修改。
VisPhyWorld: Probing Physical Reasoning via Code-Driven Video Reconstruction
- 要求模型从视觉输入生成可运行的模拟器代码，直接暴露世界表征
- 在209个场景中97.7%的重建视频有效，但多数模型无法准确推断物理参数
- 适合关注物理推理真实性的研究者，尤其用于检测模型是否真懂物理
评估多模态大语言模型（MLLMs）是否真正理解物理动态仍具挑战。现有基准多依赖识别类任务如视觉问答（VQA）和预期违背（VoE），这些任务常可在不建立可检验物理假设的前提下完成。我们提出VisPhyWorld，一种基于执行的评估框架，要求模型从视觉观察生成可执行的模拟器代码。通过生成可运行代码，推断的世界表征变得可检查、可编辑、可证伪，从而将物理推理与渲染过程分离。在此框架基础上，我们构建了VisPhyBench，包含209个由108个物理模板生成的评测场景及系统性协议，评估模型在外观重建与物理合理运动再现方面的能力。实验表明，尽管当前最优的MLLMs具备较强的语义场景理解能力，但在推断物理参数和模拟一致物理动态方面仍表现不佳。代码已开源：https://github.com/TIGER-AI-Lab/VisPhyWorld
原文摘要 · Abstract (English)
Evaluating whether Multimodal Large Language Models (MLLMs) genuinely reason about physical dynamics remains challenging. Most existing benchmarks rely on recognition-style protocols such as Visual Question Answering (VQA) and Violation of Expectation (VoE), which can often be answered without committing to an explicit, testable physical hypothesis. We propose VisPhyWorld, an execution-based framework that evaluates physical reasoning by requiring models to generate executable simulator code from visual observations. By producing runnable code, the inferred world representation is directly inspectable, editable, and falsifiable. This separates physical reasoning from rendering. Building on this framework, we introduce VisPhyBench, comprising 209 evaluation scenes derived from 108 physical templates and a systematic protocol that evaluates how well models reconstruct appearance and reproduce physically plausible motion. Our pipeline produces valid reconstructed videos in 97.7% of benchmark runs before fallback. Experiments show that while state-of-the-art MLLMs achieve strong semantic scene understanding, they struggle to accurately infer physical parameters and to simulate consistent physical dynamics. Our code is available https://github.com/TIGER-AI-Lab/VisPhyWorld
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。