用视频生成模型给静态人形网格加动态，低成本实现逼真4D动画。
Animating the Uncaptured: Humanoid Mesh Animation with Video Diffusion Models
- 利用视频生成模型的运动先验，从文本和静态网格生成动画视频。
- 通过SMPL优化将视频动作迁移到3D网格，实现真实人体运动。
- 无需复杂建模，适合快速生成多样人形动画的创作者。
人形角色动画在图形应用中至关重要，但制作高质量动画耗时且成本高。本文提出一种方法，可对输入的静态3D人形网格合成4D动画序列，借助生成式视频模型中的强泛化运动先验——这些模型蕴含丰富的人体运动信息。给定一个静态3D人形网格和描述期望动画的文本提示，我们生成一个以网格渲染图为条件的视频。随后，基于底层的SMPL表示，通过运动优化将视频生成的动作映射到对应3D网格上。该方法实现了低成本、易访问的多样化与逼真4D动画合成。
原文摘要 · Abstract (English)
Animation of humanoid characters is essential in various graphics applications, but requires significant time and cost to create realistic animations. We propose an approach to synthesize 4D animated sequences of input static 3D humanoid meshes, leveraging strong generalized motion priors from generative video models -- as such video models contain powerful motion information covering a wide variety of human motions. From an input static 3D humanoid mesh and a text prompt describing the desired animation, we synthesize a corresponding video conditioned on a rendered image of the 3D mesh. We then employ an underlying SMPL representation to animate the corresponding 3D mesh according to the video-generated motion, based on our motion optimization. This enables a cost-effective and accessible solution to enable the synthesis of diverse and realistic 4D animations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。