用物理仿真生成通用人物-物体交互数据,提升模型泛化与物理合理性。
Policy-as-Data: Learning Generalizable HOI Diffusion Models from Simulated Physics

- 通过强化学习训练策略生成任务导向的合成交互数据
- 在未见物体上实现长时序、高动态多样性交互生成
- 适合需要真实感虚拟角色和场景的开发人员
真实人物-物体交互的合成对创建具身化身和功能性虚拟环境至关重要。然而,现有数据驱动方法主要依赖昂贵且功能多样的运动捕捉数据集,训练模型难以泛化到未见物体,且长期交互中物理一致性不足。本文提出一种新框架,利用物理模拟器克服数据稀缺问题。具体而言,我们设计了一个可扩展的流程 ours,通过在物理模拟器中使用强化学习训练策略生成任务导向的数据,并基于扩充数据集训练生成模型以实现通用的HOI生成。为有效利用合成数据,我们引入粗到细的重定向过程,弥合简化模拟模型与标准参数化人体模型之间的表征差距。综合实验验证表明,该方法在未见物体上具备更强泛化能力,可生成长时序交互,同时展现出更高的动态多样性和物理合理性。
原文摘要 · Abstract (English)
Synthesizing realistic Human-Object Interactions (HOI) is critical for creating embodied avatars and functional virtual environments. However, current data-driven approaches primarily rely on motion capture datasets, which are expensive to scale and limited in functional diversity. Models trained with these datasets fail to generalize to unseen objects and maintain physical consistency over long horizons. In this paper, we propose a novel framework that leverages a physics simulator to overcome the data-scarcity bottleneck in HOI generation. Specifically, we propose a scalable pipeline, called \ours, which leverages policies trained with reinforcement learning in a physics simulator for task-oriented data generation and trains a generative model on the augmented dataset for generalizable HOI generation. To seamlessly utilize the synthetic data, we introduce a coarse-to-fine retargeting process that bridges the representation gap between the simplified model used in physics simulator and the standard parametric body models required for generative training. Validated through comprehensive experiments, our method demonstrates enhanced generalization to unseen objects and the capability of long-horizon generation, while exhibiting greater dynamic diversity and physical plausibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。