用背景条件控制角色动画，实现复杂场景下的自然动作生成。
AniCrafter: Customizing Realistic Human-Centric Animation via Avatar-Background Conditioning in Video Diffusion Models
- 通过角色与背景联合建模，将动画转为可修复问题。
- 在动态背景中生成更自然的动作，显著优于现有方法。
- 适合需要真实互动的影视、游戏角色动画开发。
视频扩散模型的进展显著提升了角色动画能力。然而，现有方法主要依赖结构化条件（如 DWPose 或 SMPL-X）来驱动角色图像动画，在涉及动态背景或复杂角色-场景交互的开放域场景中表现受限。本文提出 AniCrafter，一种基于扩散模型的人类中心动画系统，能够在开放域动态背景中无缝集成并动画化指定角色，同时遵循预设的人体运动序列。该模型基于先进的 Image-to-Video（I2V）扩散架构，引入创新的“角色-背景”联合条件机制，将开放域人机中心动画重构为一个恢复问题，从而实现多样化且具备遮挡感知能力的动画结果。实验表明，该方法超越当前最优水平，尤其在挑战性场景中表现出色。代码与模型已开源：https://github.com/MyNiuuu/AniCrafter。
原文摘要 · Abstract (English)
Recent advances in video diffusion models have substantially enhanced character animation techniques. However, existing methods primarily depend on structural conditions, such as DWPose or SMPL-X, to animate character images, which limits their effectiveness in open-domain scenarios involving dynamic backgrounds or complex character-scene interactions. This study presents AniCrafter, a diffusion-based human-centric animation model designed to seamlessly integrate and animate a given character within open-domain dynamic backgrounds while adhering to specified human motion sequences. Built upon advanced Image-to-Video (I2V) diffusion architectures, the model introduces an innovative "avatar-background" conditioning mechanism that reformulates open-domain human-centric animation as a restoration problem, thereby achieving versatile, occlusion-aware animation results. Experimental evaluations demonstrate that the proposed approach outperforms current state-of-the-art methods and exhibits an exceptional capability in handling challenging scenarios. Codes and model are available at: https://github.com/MyNiuuu/AniCrafter
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。