无需模板和绑定即可实现跨角色动画迁移,支持新动作与新体型的泛化。
SMF: Template-free and Rig-free Animation Transfer using Kinetic Codes
- 用自监督方式学习运动编码,仅需稀疏运动数据训练。
- 在AMASS等数据集上实现新动作与新角色的最优泛化性能。
- 适合需要快速迁移动画的工业场景,如游戏与影视制作。
动画重定向将稀疏运动描述(如关键点序列)映射到角色网格,生成语义合理且时间连贯的全身网格序列。现有方法受限于需模板形状先验或人工设计的变形绑定,泛化能力差或存在运动抖动。本文提出自监督运动场(SMF),仅依赖稀疏运动表示进行训练,无需数据集特定标注、模板或绑定。核心是基于自编码器的运动编码——动能码(Kinetic Codes),其揭示了富含语义的潜在空间,便于大规模训练。网络包含专用的空间与时间梯度预测模块,端到端联合训练,并由动能码潜在空间正则化,从而在未见角色与新动作间具有良好泛化性。我们在AMASS、D4D、Mixamo及单目视频中采样的未见动作上评估,对多种形状与拓扑的角色进行动画迁移。在未见动作泛化任务上,AMASS数据集达到当前最优表现。代码、权重与补充材料详见项目主页:https://motionfields.github.io/
原文摘要 · Abstract (English)
Animation retargetting applies sparse motion description (e.g., keypoint sequences) to a character mesh to produce a semantically plausible and temporally coherent full-body mesh sequence. Existing approaches come with restrictions -- they require access to template-based shape priors or artist-designed deformation rigs, suffer from limited generalization to unseen motion and/or shapes, or exhibit motion jitter. We propose Self-supervised Motion Fields (SMF), a self-supervised framework that is trained with only sparse motion representations, without requiring dataset-specific annotations, templates, or rigs. At the heart of our method are Kinetic Codes, a novel autoencoder-based sparse motion encoding, that exposes a semantically rich latent space, simplifying large-scale training. Our architecture comprises dedicated spatial and temporal gradient predictors, which are jointly trained in an end-to-end fashion. The combined network, regularized by the Kinetic Codes' latent space, has good generalization across both unseen shapes and new motions. We evaluated our method on unseen motion sampled from AMASS, D4D, Mixamo, and raw monocular video for animation transfer on various characters with varying shapes and topology. We report a new SoTA on the AMASS dataset in the context of generalization to unseen motion. Code, weights, and supplementary are available on the project webpage at https://motionfields.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。