用语义与物理双系统生成无碰撞的桌面场景,让机器人仿真更真实。
STABLE: Simulation-Ready Tabletop Layout Generation via a Semantics-Physics Dual System

- 先用语言模型生成粗略布局,再用物理模型修正位置消除碰撞。
- 生成的场景符合任务指令且物理上合理,相比旧方法提升显著。
- 适合需要真实仿真环境的机器人任务设计与测试人群。
从任务指令生成可直接用于仿真的桌面场景是具身智能领域的一个重要方向。然而,现有方法仅依赖大语言模型预测布局,常因三维空间推理能力不足导致物体重叠或漂浮。本文提出STABLE,一种面向仿真可用桌面场景生成的语义-物理双系统。该系统包含两个互补模块:(i) 语义推理器,基于结构化桌面场景数据微调的语言模型,从任务指令生成粗略布局;(ii) 物理校正器,一种感知物理规律的基于流的去噪模型,输出姿态更新以优化布局,确保物理合理性同时保持与任务指令的语义一致。STABLE采用渐进式生成范式:通过交替使用语义推理器与物理校正器,逐步从任务关键物体扩展至背景物体。实验表明,STABLE能生成严格遵循任务指令且显著提升物理合理性的仿真可用桌面场景,优于现有方法。
原文摘要 · Abstract (English)
Generating simulation-ready tabletop scenes from task instructions is an intriguing and promising research direction in the field of Embodied AI. However, existing task-to-scene generation methods rely exclusively on large language models (LLMs) to predict scene layouts, inevitably yielding object collisions or floating due to LLMs' inherent limitations in 3D spatial reasoning. In this paper, we present STABLE, a semantics-physics dual-system tailored for simulation-ready tabletop scene generation. STABLE consists of two complementary modules: (i) a Semantic Reasoner, a fine-tuned LLM trained on a structured tabletop scene dataset to generate coarse layouts from input task instructions, and (ii) a Physics Corrector, a physics-aware flow-based denoising model that outputs pose updates to refine layouts, which ensures the physical plausibility of scenes while preserves semantic alignment with task instructions. STABLE adopts a progressive generation paradigm: by alternating between the Semantic Reasoner and Physics Corrector, it incrementally expands the scene from task-critical objects to background objects. Experiments demonstrate that STABLE successfully generates simulation-ready tabletop scenes that strictly conform to task instructions and significantly enhances the physical validity of scenes over prior art.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。