让3D场景可被机器人真实行走操作，同时支持多种实体交互。
RoboLayout: Differentiable 3D Scene Generation for Embodied Agents
- 通过可微优化加入可达性约束，生成可行动的3D布局。
- 局部重优化提升收敛效率，不增加全局迭代次数。
- 适配不同物理能力的实体，如人、服务机器人、动物等。
视觉语言模型（VLM）在从开放语言指令生成3D场景布局方面展现出强大潜力。然而，在物理受限的室内环境中，生成既语义一致又可供具身智能体交互的布局仍具挑战。本文提出RoboLayout，作为LayoutVLM的扩展，引入了代理感知推理与更稳定的优化机制。该方法将显式的可达性约束融入可微布局优化过程，使生成的场景具备可导航性和可操作性。重要的是，代理抽象不限于特定机器人平台，可表示具有不同物理能力的各类实体，如服务机器人、仓储机器人、不同年龄人群或动物，实现面向特定代理的环境设计。此外，提出局部精修阶段，选择性地重新优化问题区域对象位置，保持其余场景固定，提升收敛效率而不增加全局优化迭代次数。实验结果表明，RoboLayout在多样场景配置下保持了LayoutVLM的强语义对齐与物理合理性，显著增强了面向代理的室内场景生成实用性。
原文摘要 · Abstract (English)
Recent advances in vision language models (VLMs) have shown strong potential for spatial reasoning and 3D scene layout generation from open-ended language instructions. However, generating layouts that are not only semantically coherent but also feasible for interaction by embodied agents remains challenging, particularly in physically constrained indoor environments. In this paper, RoboLayout is introduced as an extension of LayoutVLM that augments the original framework with agent-aware reasoning and improved optimization stability. RoboLayout integrates explicit reachability constraints into a differentiable layout optimization process, enabling the generation of layouts that are navigable and actionable by embodied agents. Importantly, the agent abstraction is not limited to a specific robot platform and can represent diverse entities with distinct physical capabilities, such as service robots, warehouse robots, humans of different age groups, or animals, allowing environment design to be tailored to the intended agent. In addition, a local refinement stage is proposed that selectively reoptimizes problematic object placements while keeping the remainder of the scene fixed, improving convergence efficiency without increasing global optimization iterations. Overall, RoboLayout preserves the strong semantic alignment and physical plausibility of LayoutVLM while enhancing applicability to agent-centric indoor scene generation, as demonstrated by experimental results across diverse scene configurations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。