用1D隐式运动令牌实现身份与动作解耦的动画生成
IM-Animation: An Implicit Motion Representation for Identity-decoupled Character Animation
- 将每帧运动压缩为紧凑的1D运动令牌,避免空间错位问题
- 在多个数据集上优于或媲美当前最佳方法,动作一致性更强
- 适合需要高保真动作迁移的数字人动画开发人员
视频扩散模型的进展显著推动了角色动画技术的发展,该技术通过驱动视频对静态身份图像进行动态化处理生成带动作的视频。显式方法使用骨骼、DWPose等结构化信号表示运动,但难以应对空间错配和身体比例差异。隐式方法则直接从驱动视频中捕捉高层次的隐式运动语义,但存在身份信息泄露和运动与外观纠缠的问题。为此,本文提出一种新型隐式运动表示,将每帧运动压缩为紧凑的1D运动令牌,缓解2D表示固有的严格空间约束,并有效防止运动视频中的身份信息泄露。此外,设计基于时间一致掩码令牌的重定向模块,通过时间训练瓶颈机制,降低源图像运动干扰,提升重定向一致性。采用三阶段训练策略提高训练效率并保障生成质量。大量实验表明,所提出的隐式运动表示及IM-Animation在生成能力上达到或超越当前最优方法。
原文摘要 · Abstract (English)
Recent progress in video diffusion models has markedly advanced character animation, which synthesizes motioned videos by animating a static identity image according to a driving video. Explicit methods represent motion using skeleton, DWPose or other explicit structured signals, but struggle to handle spatial mismatches and varying body scales. %proportions. Implicit methods, on the other hand, capture high-level implicit motion semantics directly from the driving video, but suffer from identity leakage and entanglement between motion and appearance. To address the above challenges, we propose a novel implicit motion representation that compresses per-frame motion into compact 1D motion tokens. This design relaxes strict spatial constraints inherent in 2D representations and effectively prevents identity information leakage from the motion video. Furthermore, we design a temporally consistent mask token-based retargeting module that enforces a temporal training bottleneck, mitigating interference from the source images' motion and improving retargeting consistency. Our methodology employs a three-stage training strategy to enhance the training efficiency and ensure high fidelity. Extensive experiments demonstrate that our implicit motion representation and the propose IM-Animation's generative capabilities are achieve superior or competitive performance compared with state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。