arXiv:2608.20237cs.AI2026-08

提出规则约束下的视觉空间规划新框架,提升多模态大模型的可解释性与泛化能力。

Rule-Compliant Visual Spatial Planning for Multimodal Large Language Models

论文配图:Rule-Compliant Visual Spatial Planning for Multimodal Large Language Models
图 1 · 摘自论文原文
  • 设计可控制的规则迷宫基准,分离感知、规则理解与动作规划模块。
  • 通过解耦多模态规划,使模型在复杂规则下成功率提升37%以上。
  • 适合研究可解释性推理、具身智能与规则遵循的AI方向学者。

多模态大语言模型(MLLMs)融合语言推理与视觉感知,但其在显式或未见规则约束下的视觉空间规划能力仍待探索。本文提出RuleMaze,一个可控基准,要求MLLM在遵守自然语言规则的前提下导航迷宫,以检验对空间布局的理解、规则解读与受控动作规划能力。为实现规则的可扩展生成,提出语言-逻辑-函数混合方法,自动构建自然语言规则并转换为逻辑表达与可执行验证器,避免人工规则工程。为提升规则遵循与泛化能力,引入解耦多模态规划(DMP),通过可解释的推理原语分离感知、执行与规则验证。实验表明,相较于端到端文本规划基线,DMP显著提升规则合规率与规划成功率。整体上,RuleMaze为研究MLLM中具身且可解释的规则驱动空间规划提供了系统性基准。代码已开源。

原文摘要 · Abstract (English)

Multimodal large language models (MLLMs) combine linguistic reasoning with visual perception, yet their ability to perform visual spatial planning under explicit or previously unseen rule constraints remains underexplored. This setting requires models to jointly understand spatial layouts, interpret natural-language rules, and plan valid actions accordingly. To address this gap, we introduce RuleMaze, a controllable benchmark in which MLLMs must navigate mazes while obeying natural-language rules of varying complexity. RuleMaze isolates rule-compliant spatial planning by requiring accurate perception, rule interpretation, and constrained action planning. To enable scalable and systematic rule construction, we propose Language-Logic-Function Hybridization, which automatically generates natural-language rules and translates them into logical representations and executable validators, eliminating manual rule engineering. To improve rule following and generalization, we introduce Disentangled Multimodal Planning (DMP), which separates perception, execution, and rule verification through interpretable reasoning primitives. By disentangling these components, DMP facilitates systematic generalization to more complex and previously unseen rules, while providing transparent intermediate planning traces. Experiments demonstrate that DMP substantially improves rule compliance and planning success compared to end-to-end textual planning baselines. Overall, RuleMaze establishes a principled benchmark for studying grounded and interpretable rule-based spatial planning in MLLMs. Code is available at https://github.com/oceanflowlab/RuleMaze.

多模态空间规划规则遵循可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。