Kling-MotionControl实现高保真角色动画,精准控制全身动作与表情。
Kling-MotionControl Technical Report
- 分治策略融合身体、面部、手部不同运动表示,兼顾结构稳定与细节表现。
- 支持跨身份自然迁移,保持角色外观一致,推理速度提升10倍以上。
- 可灵活响应文本指令,适合影视、游戏等需要精细动画控制的场景。
角色动画旨在通过将驱动视频中的动态信息迁移至参考图像,生成逼真的视频。本文提出Kling-MotionControl,一种基于DiT的统一框架,专为鲁棒、精确且富有表现力的整体角色动画设计。该模型采用分治策略,在统一系统内针对身体、面部和手部的不同特性,协同处理异构运动表示,有效平衡大尺度结构稳定性与细微关节表达性。为确保跨身份泛化能力,引入自适应无身份学习机制,实现从真实人类到卡通风格角色的自然动作重定向。同时,通过精细的身份注入与融合设计,保障外观忠实还原,并辅以主体库机制,利用全面的参考上下文增强效果。为提升实用性,采用多阶段蒸馏加速框架,推理速度提升超过10倍。人类偏好评估表明,Kling-MotionControl在整体运动控制、开放域泛化能力以及视觉质量与连贯性方面均优于主流商业及开源方案,展现出卓越性能,确立其在高质量、可控且逼真角色动画领域的领先地位。
原文摘要 · Abstract (English)
Character animation aims to generate lifelike videos by transferring motion dynamics from a driving video to a reference image. Recent strides in generative models have paved the way for high-fidelity character animation. In this work, we present Kling-MotionControl, a unified DiT-based framework engineered specifically for robust, precise, and expressive holistic character animation. Leveraging a divide-and-conquer strategy within a cohesive system, the model orchestrates heterogeneous motion representations tailored to the distinct characteristics of body, face, and hands, effectively reconciling large-scale structural stability with fine-grained articulatory expressiveness. To ensure robust cross-identity generalization, we incorporate adaptive identity-agnostic learning, facilitating natural motion retargeting for diverse characters ranging from realistic humans to stylized cartoons. Simultaneously, we guarantee faithful appearance preservation through meticulous identity injection and fusion designs, further supported by a subject library mechanism that leverages comprehensive reference contexts. To ensure practical utility, we implement an advanced acceleration framework utilizing multi-stage distillation, boosting inference speed by over 10x. Kling-MotionControl distinguishes itself through intelligent semantic motion understanding and precise text responsiveness, allowing for flexible control beyond visual inputs. Human preference evaluations demonstrate that Kling-MotionControl delivers superior performance compared to leading commercial and open-source solutions, achieving exceptional fidelity in holistic motion control, open domain generalization, and visual quality and coherence. These results establish Kling-MotionControl as a robust solution for high-quality, controllable, and lifelike character animation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。