构建真实物理与视觉的机器人操作仿真环境，支持长程任务规划研究
MuBlE: MuJoCo and Blender simulation Environment and Benchmark for Task Planning in Robot Manipulation
- 基于MuJoCo和Blender构建高保真仿真环境，支持物理与视觉双重反馈
- 首次实现长程机器人操作任务的精准物理建模与多模态数据生成
- 适用于需要视觉与物理协同推理的复杂操作任务研究
当前具身推理智能体在规划需与世界物理交互以获取信息的长程任务（如‘将物体按重量从轻到重排序’）时面临困难。其能力提升高度依赖于高质量训练环境。为此，我们提出一个新仿真环境MuBlE，基于robosuite构建，采用MuJoCo物理引擎与Blender高精度渲染器，提供真实视觉观测并保持场景物理状态精确一致。它是首个专注于长程机器人操作任务、同时保证物理建模准确性的仿真平台。MuBlE可生成多模态数据用于训练，并通过两个层面的闭环机制支持方法设计：视觉-动作回路与控制-物理回路。同时，我们提出SHOP-VRB2基准，包含10类多步推理场景，要求同时依赖视觉与物理测量。
原文摘要 · Abstract (English)
Current embodied reasoning agents struggle to plan for long-horizon tasks that require to physically interact with the world to obtain the necessary information (e.g. 'sort the objects from lightest to heaviest'). The improvement of the capabilities of such an agent is highly dependent on the availability of relevant training environments. In order to facilitate the development of such systems, we introduce a novel simulation environment (built on top of robosuite) that makes use of the MuJoCo physics engine and high-quality renderer Blender to provide realistic visual observations that are also accurate to the physical state of the scene. It is the first simulator focusing on long-horizon robot manipulation tasks preserving accurate physics modeling. MuBlE can generate mutlimodal data for training and enable design of closed-loop methods through environment interaction on two levels: visual - action loop, and control - physics loop. Together with the simulator, we propose SHOP-VRB2, a new benchmark composed of 10 classes of multi-step reasoning scenarios that require simultaneous visual and physical measurements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。