arXiv:2508.14120cs.ROcs.AI2025-08被引 5

用生成模型与强化学习结合,让机器人动作更真实自然。

SimGenHOI: Physically Realistic Whole-Body Humanoid-Object Interaction via Generative Modeling and Reinforcement Learning

  • 用扩散Transformer生成关键动作,再插值成平滑轨迹。
  • 强化学习控制策略纠正穿模、脚滑等物理错误,成功率显著提升。
  • 生成模型与控制策略互训优化,适合长时序机械操作任务。

生成物理上真实的类人机器人-物体交互(HOI)是机器人领域的重要挑战。现有基于扩散模型的方法常出现不合理的接触、穿透和非自然全身动作,难以在真实环境中执行。为此,我们提出SimGenHOI,一个融合生成建模与强化学习的统一框架,实现可控且物理真实的HOI生成。我们的HOI生成模型基于扩散Transformer(DiT),根据文本提示、物体几何形状、稀疏物体路径点和初始人体姿态预测一组关键动作,并通过插值生成平滑运动轨迹,天然支持长时序生成。为确保物理真实性,我们设计了基于强化学习的接触感知全身控制策略,可追踪生成动作并修正穿透、脚滑等缺陷。此外,我们引入双向微调机制,使生成模型与控制策略相互优化,提升动作真实性和跟踪鲁棒性。大量实验表明,SimGenHOI生成的动作具有高真实性、多样性和物理合理性,在仿真中显著提升跟踪成功率,成功完成长时序操作任务。代码将在论文接收后公开于项目页:https://xingxingzuo.github.io/simgen_hoi。

原文摘要 · Abstract (English)

Generating physically realistic humanoid-object interactions (HOI) is a fundamental challenge in robotics. Existing HOI generation approaches, such as diffusion-based models, often suffer from artifacts such as implausible contacts, penetrations, and unrealistic whole-body actions, which hinder successful execution in physical environments. To address these challenges, we introduce SimGenHOI, a unified framework that combines the strengths of generative modeling and reinforcement learning to produce controllable and physically plausible HOI. Our HOI generative model, based on Diffusion Transformers (DiT), predicts a set of key actions conditioned on text prompts, object geometry, sparse object waypoints, and the initial humanoid pose. These key actions capture essential interaction dynamics and are interpolated into smooth motion trajectories, naturally supporting long-horizon generation. To ensure physical realism, we design a contact-aware whole-body control policy trained with reinforcement learning, which tracks the generated motions while correcting artifacts such as penetration and foot sliding. Furthermore, we introduce a mutual fine-tuning strategy, where the generative model and the control policy iteratively refine each other, improving both motion realism and tracking robustness. Extensive experiments demonstrate that SimGenHOI generates realistic, diverse, and physically plausible humanoid-object interactions, achieving significantly higher tracking success rates in simulation and enabling long-horizon manipulation tasks. Code will be released upon acceptance on our project page: https://xingxingzuo.github.io/simgen_hoi.

机器人交互生成模型强化学习动作生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。