让游戏角色也能自然动起来,动画效果更真实。
Animate-X: Universal Character Image Animation with Enhanced Motion Representation
- 用视觉特征+预演输入增强动作表征,捕捉驱动视频的运动模式。
- 在多种角色上表现优异,尤其对类人角色泛化能力强。
- 专设测试集评估,适合游戏影视行业应用。
角色图像动画近年取得显著进展,但多数方法仅适用于人类形象,难以泛化到游戏与娱乐行业中常见的类人角色。深入分析表明,根源在于现有方法对运动建模不足,无法理解驱动视频的动作模式,导致将姿态序列僵硬套用至目标角色。为此,本文提出通用动画框架Animate-X,基于潜在扩散模型(LDM)支持各类角色(统称X)的动画生成。为增强运动表征,引入姿态指示器(Pose Indicator),通过隐式与显式方式捕捉驱动视频的完整运动模式:前者利用CLIP视觉特征提取整体运动趋势与时间关联,后者通过预演推理可能输入提升LDM泛化能力。此外,构建新基准A^2Bench以评估在通用、广泛适用图像上的动画性能。大量实验验证了Animate-X在主流方法中的优越性。
原文摘要 · Abstract (English)
Character image animation, which generates high-quality videos from a reference image and target pose sequence, has seen significant progress in recent years. However, most existing methods only apply to human figures, which usually do not generalize well on anthropomorphic characters commonly used in industries like gaming and entertainment. Our in-depth analysis suggests to attribute this limitation to their insufficient modeling of motion, which is unable to comprehend the movement pattern of the driving video, thus imposing a pose sequence rigidly onto the target character. To this end, this paper proposes Animate-X, a universal animation framework based on LDM for various character types (collectively named X), including anthropomorphic characters. To enhance motion representation, we introduce the Pose Indicator, which captures comprehensive motion pattern from the driving video through both implicit and explicit manner. The former leverages CLIP visual features of a driving video to extract its gist of motion, like the overall movement pattern and temporal relations among motions, while the latter strengthens the generalization of LDM by simulating possible inputs in advance that may arise during inference. Moreover, we introduce a new Animated Anthropomorphic Benchmark (A^2Bench) to evaluate the performance of Animate-X on universal and widely applicable animation images. Extensive experiments demonstrate the superiority and effectiveness of Animate-X compared to state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。