arXiv:2505.13437cs.CVcs.AI2025-05CVPR被引 15

用物理定律生成更逼真的精细动作,提升人体运动自然度。

FinePhys: Fine-grained Human Action Generation by Explicitly Incorporating Physical Laws for Effective Skeletal Guidance

  • 结合物理方程实时优化3D骨骼姿态,提升动作合理性。
  • 在FineGym数据集上显著优于现有方法,尤其在翻转、跳跃等复杂动作中。
  • 适合需要高精度人体动作生成的研究者与应用开发人员。

尽管视频生成技术进展显著,但生成符合物理规律的人体动作仍面临挑战,尤其是在建模精细语义和复杂时间动态方面。例如,生成如‘0.5圈转体跳’这类体操动作对现有方法而言仍具难度,常导致结果不理想。为此,我们提出FinePhys,一种通过显式引入物理规律实现精细化人体动作生成的框架,以获得有效的骨骼引导。具体而言,FinePhys首先在线估计2D姿态,再通过上下文学习完成2D到3D的姿态重建。为缓解纯数据驱动3D姿态的不稳定性和可解释性差问题,我们引入基于欧拉-拉格朗日方程的物理运动重估模块,通过双向时间更新计算关节加速度。物理预测的3D姿态与数据驱动结果融合,为扩散过程提供多尺度2D热图引导。在FineGym数据集的三个细粒度动作子集(FX-JUMP、FX-TURN、FX-SALTO)上评估,FinePhys显著优于对比基线。定性结果进一步表明其能生成更自然、更合理的细粒度人体动作。

原文摘要 · Abstract (English)

Despite significant advances in video generation, synthesizing physically plausible human actions remains a persistent challenge, particularly in modeling fine-grained semantics and complex temporal dynamics. For instance, generating gymnastics routines such as "switch leap with 0.5 turn" poses substantial difficulties for current methods, often yielding unsatisfactory results. To bridge this gap, we propose FinePhys, a Fine-grained human action generation framework that incorporates Physics to obtain effective skeletal guidance. Specifically, FinePhys first estimates 2D poses in an online manner and then performs 2D-to-3D dimension lifting via in-context learning. To mitigate the instability and limited interpretability of purely data-driven 3D poses, we further introduce a physics-based motion re-estimation module governed by Euler-Lagrange equations, calculating joint accelerations via bidirectional temporal updating. The physically predicted 3D poses are then fused with data-driven ones, offering multi-scale 2D heatmap guidance for the diffusion process. Evaluated on three fine-grained action subsets from FineGym (FX-JUMP, FX-TURN, and FX-SALTO), FinePhys significantly outperforms competitive baselines. Comprehensive qualitative results further demonstrate FinePhys's ability to generate more natural and plausible fine-grained human actions.

动作生成物理引导骨骼重建扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。