让角色动画更自然,能合理互动环境。
Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance
- 用环境特征作条件,让角色动画贴合场景
- 新掩码策略提升角色与环境关系表达
- 适合需要真实场景交互的动画应用
基于扩散模型的近期角色图像动画方法(如 Animate Anyone)在生成一致且可泛化的角色动画方面取得了显著进展。然而,这些方法无法生成角色与环境之间的合理关联。为此,我们提出 Animate Anyone 2,旨在实现带有环境可及性(environment affordance)的角色图像动画。除了从源视频中提取运动信号外,我们还额外捕捉环境表征作为条件输入。将环境定义为排除角色后的区域,模型生成角色以填充这些区域,并保持与环境上下文的一致性。我们提出一种无形状依赖的掩码策略,更有效地刻画角色与环境的关系。为进一步提升物体交互的真实性,我们引入物体引导器提取交互物体特征,并采用空间融合方式注入特征。此外,还设计了一种姿态调制策略,使模型能够处理更丰富的运动模式。实验结果表明,所提方法性能优越。
原文摘要 · Abstract (English)
Recent character image animation methods based on diffusion models, such as Animate Anyone, have made significant progress in generating consistent and generalizable character animations. However, these approaches fail to produce reasonable associations between characters and their environments. To address this limitation, we introduce Animate Anyone 2, aiming to animate characters with environment affordance. Beyond extracting motion signals from source video, we additionally capture environmental representations as conditional inputs. The environment is formulated as the region with the exclusion of characters and our model generates characters to populate these regions while maintaining coherence with the environmental context. We propose a shape-agnostic mask strategy that more effectively characterizes the relationship between character and environment. Furthermore, to enhance the fidelity of object interactions, we leverage an object guider to extract features of interacting objects and employ spatial blending for feature injection. We also introduce a pose modulation strategy that enables the model to handle more diverse motion patterns. Experimental results demonstrate the superior performance of the proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。