用自引导随机采样实现逼真3D人体动画,解决动作失真与身份丢失问题
Ani3DHuman: Photorealistic 3D Human Animation with Self-guided Stochastic Sampling

- 分层运动表征分离刚体与非刚体动作,结合动力学与扩散模型
- 自引导随机采样使初始渲染异常时仍能生成高质量视频,避免身份丢失
- 适合做高保真3D角色动画、影视特效的开发者和研究者
当前3D人体动画方法难以实现逼真效果:基于运动学的方法缺乏非刚性动态(如衣物运动),而利用视频扩散先验的方法虽可生成非刚性运动,却存在质量瑕疵与身份丢失问题。为此,我们提出Ani3DHuman框架,融合运动学动画与视频扩散先验。首先引入分层运动表征,将刚体运动与残差非刚体运动解耦;刚体运动由运动学方法生成粗略渲染,引导视频扩散模型恢复残差非刚体运动。然而,该恢复任务在扩散采样中极具挑战,因初始渲染分布外,标准确定性ODE采样器失效。为此,我们提出新颖的自引导随机采样方法,通过结合随机采样(保证逼真度)与自引导(保持身份一致性),有效解决分布外问题。生成的高质量视频提供监督信号,用于优化残差非刚体运动场。大量实验表明, MethodName能生成逼真3D人体动画,优于现有方法。代码已公开于https://github.com/qiisun/ani3dhuman。
原文摘要 · Abstract (English)
Current 3D human animation methods struggle to achieve photorealism: kinematics-based approaches lack non-rigid dynamics (e.g., clothing dynamics), while methods that leverage video diffusion priors can synthesize non-rigid motion but suffer from quality artifacts and identity loss. To overcome these limitations, we present Ani3DHuman, a framework that marries kinematics-based animation with video diffusion priors. We first introduce a layered motion representation that disentangles rigid motion from residual non-rigid motion. Rigid motion is generated by a kinematic method, which then produces a coarse rendering to guide the video diffusion model in generating video sequences that restore the residual non-rigid motion. However, this restoration task, based on diffusion sampling, is highly challenging, as the initial renderings are out-of-distribution, causing standard deterministic ODE samplers to fail. Therefore, we propose a novel self-guided stochastic sampling method, which effectively addresses the out-of-distribution problem by combining stochastic sampling (for photorealistic quality) with self-guidance (for identity fidelity). These restored videos provide high-quality supervision, enabling the optimization of the residual non-rigid motion field. Extensive experiments demonstrate that \MethodName can generate photorealistic 3D human animation, outperforming existing methods. Code is available in https://github.com/qiisun/ani3dhuman.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。