arXiv:2510.07882cs.RO2025-10被引 2

让双臂人形机器人更懂身体位置,提升复杂任务规划能力。

Towards Proprioception-Aware Embodied Planning for Dual-Arm Humanoid Robots

  • 引入本体感知信息与运动嵌入,增强模型对身体位置的理解。
  • 在新模拟器上,任务成功率比现有模型平均提升19.75%。
  • 适合研究具身智能、双臂协作或人形机器人规划的团队使用。

近年来,多模态大语言模型(MLLMs)展现出作为高层规划器的能力,使机器人能够执行复杂的人类指令。然而,在涉及双臂人形机器人的长时序任务中,其效果仍受限。主要挑战有二:(i) 缺乏系统支持人形机器人任务评估与数据收集的仿真平台;(ii) 当前MLLMs的具身感知不足,难以推理双臂选择逻辑与躯体姿态。为此,我们提出DualTHOR——一个支持连续状态转移与异常处理机制的双臂人形机器人仿真环境。基于此平台,我们构建Proprio-MLLM,通过融合本体感知信息、基于运动的位置嵌入与跨空间编码器,提升模型的具身意识。实验表明,尽管现有MLLMs在此环境中表现不佳,Proprio-MLLM在规划性能上实现平均19.75%的提升。本工作同时提供了关键的仿真平台与有效模型,推动人形机器人具身智能的发展。代码已公开于https://anonymous.4open.science/r/DualTHOR-5F3B。

原文摘要 · Abstract (English)

In recent years, Multimodal Large Language Models (MLLMs) have demonstrated the ability to serve as high-level planners, enabling robots to follow complex human instructions. However, their effectiveness, especially in long-horizon tasks involving dual-arm humanoid robots, remains limited. This limitation arises from two main challenges: (i) the absence of simulation platforms that systematically support task evaluation and data collection for humanoid robots, and (ii) the insufficient embodiment awareness of current MLLMs, which hinders reasoning about dual-arm selection logic and body positions during planning. To address these issues, we present DualTHOR, a new dual-arm humanoid simulator, with continuous transition and a contingency mechanism. Building on this platform, we propose Proprio-MLLM, a model that enhances embodiment awareness by incorporating proprioceptive information with motion-based position embedding and a cross-spatial encoder. Experiments show that, while existing MLLMs struggle in this environment, Proprio-MLLM achieves an average improvement of 19.75% in planning performance. Our work provides both an essential simulation platform and an effective model to advance embodied intelligence in humanoid robotics. The code is available at https://anonymous.4open.science/r/DualTHOR-5F3B.

具身智能双臂机器人大模型仿真平台

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。