arXiv:2506.16012cs.RO2025-06被引 3

DualTHOR模拟双臂机器人执行任务时的意外情况,提升模型在真实环境中的鲁棒性。

DualTHOR: A Dual-Arm Humanoid Simulation Platform for Contingency-Aware Planning

  • 基于物理引擎构建双臂人形机器人仿真平台,支持真实机器人资产和逆运动学求解
  • 引入故障机制模拟执行中的不确定性,评估模型在真实场景下的表现
  • 揭示当前视觉语言模型在双臂协作和抗干扰能力上的不足,适合研究具身智能的学者

实现复杂交互任务的具身智能体仍面临重大挑战。尽管近期仿真平台显著提升了训练任务多样性,但多数平台采用简化机器人结构并忽略底层执行的随机性,限制了向真实机器人的迁移。为此,我们提出基于扩展版AI2-THOR的物理仿真平台DualTHOR,专为复杂双臂人形机器人设计。平台包含真实机器人资产、双臂协作任务集及人形机器人的逆运动学求解器,并引入基于物理的低层执行故障机制,模拟潜在失败情况,弥合与真实场景的差距。该平台可更全面评估视觉语言模型(VLMs)在家庭环境中的鲁棒性和泛化能力。大量实验表明,当前VLMs在双臂协同方面表现不佳,且在含突发状况的真实环境中鲁棒性有限,凸显使用本平台开发更强具身智能模型的重要性。代码已开源:https://github.com/ds199895/DualTHOR.git。

原文摘要 · Abstract (English)

Developing embodied agents capable of performing complex interactive tasks in real-world scenarios remains a fundamental challenge in embodied AI. Although recent advances in simulation platforms have greatly enhanced task diversity to train embodied Vision Language Models (VLMs), most platforms rely on simplified robot morphologies and bypass the stochastic nature of low-level execution, which limits their transferability to real-world robots. To address these issues, we present a physics-based simulation platform DualTHOR for complex dual-arm humanoid robots, built upon an extended version of AI2-THOR. Our simulator includes real-world robot assets, a task suite for dual-arm collaboration, and inverse kinematics solvers for humanoid robots. We also introduce a contingency mechanism that incorporates potential failures through physics-based low-level execution, bridging the gap to real-world scenarios. Our simulator enables a more comprehensive evaluation of the robustness and generalization of VLMs in household environments. Extensive evaluations reveal that current VLMs struggle with dual-arm coordination and exhibit limited robustness in realistic environments with contingencies, highlighting the importance of using our simulator to develop more capable VLMs for embodied tasks. The code is available at https://github.com/ds199895/DualTHOR.git.

具身智能双臂协作仿真平台视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。