用折纸挑战让AI学会物理因果推理,发现大模型仍不会连贯规划折叠步骤。
OrigamiBench: An Interactive Environment to Synthesize Flat-Foldable Origamis
- 设计可交互的折纸环境,让模型逐步提出折叠动作并获反馈。
- 大模型虽强但无法生成连贯多步折叠策略,暴露其因果推理缺陷。
- 适合研究视觉语言模型、具身智能与物理推理的学者参考。
构建能规划、行动并创造物理世界的AI系统,不仅需要模式识别,更需理解物理过程背后的因果机制与约束以指导序列决策。这种能力依赖于类似内部语言模型的表征,能关联观察、动作与环境变化。然而,现有基准常将视觉感知与符号推理割裂,或专注视觉识别,或聚焦符号任务。折纸领域自然融合了多模态需求:折叠造形需视觉感知、几何与物理约束推理及序列规划,同时结构清晰,利于系统评估。我们提出OrigamiBench,一个交互式基准,模型可迭代提出折叠操作,并获得关于物理有效性及与目标构型相似度的反馈。对现代视觉-语言模型的实验表明,单纯扩大模型规模并不能可靠提升对物理变换的因果推理能力。模型难以生成连贯的多步折叠策略,暗示视觉与语言表征仍弱耦合。
原文摘要 · Abstract (English)
Building AI systems that can plan, act, and create in the physical world requires more than pattern recognition. Such systems must understand the causal mechanisms and constraints governing physical processes in order to guide sequential decisions. This capability relies on internal representations, analogous to an internal language model, that relate observations, actions, and resulting environmental changes. However, many existing benchmarks treat visual perception and programmatic reasoning as separate problems, focusing either on visual recognition or on symbolic tasks. The domain of origami provides a natural testbed that integrates these modalities. Constructing shapes through folding operations requires visual perception, reasoning about geometric and physical constraints, and sequential planning, while remaining sufficiently structured for systematic evaluation. We introduce OrigamiBench, an interactive benchmark in which models iteratively propose folds and receive feedback on physical validity and similarity to a target configuration. Experiments with modern vision-language models show that scaling model size alone does not reliably produce causal reasoning about physical transformations. Models fail to generate coherent multi-step folding strategies, suggesting that visual and language representations remain weakly integrated.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。