让人物动画中的背景自然动起来,无需摄像机轨迹
AnimateAnywhere: Rouse the Background in Human Image Animation
- 从人物姿态序列推断背景运动,实现无相机轨迹的动态背景生成
- 引入视极约束增强跨帧对应关系,避免不合理注意力分布
- 适合影视、游戏等需真实背景互动的动画创作场景
人体图像动画旨在生成符合指定姿态序列的人物视频,但现有方法多关注人物动作,忽略背景生成,导致结果静态或不协调。尽管社区探索了基于摄像机姿态的动画任务,但对多数娱乐应用和普通用户而言,手动设计摄像机轨迹不现实。为此,我们提出 AnimateAnywhere 框架,在无需摄像机轨迹的前提下激活背景动态。核心洞察是:人体运动常反映背景运动。因此,我们引入背景运动学习器(BML),从人物姿态序列中学习背景运动。为进一步提升跨帧对应准确性,我们在三维注意力图上施加视极约束:通过融合视极掩码与当前3D注意力图,构建精细掩码以抑制几何上不合理注意力。大量实验表明,AnimateAnywhere能有效从姿态序列学习背景运动,在生成具有生动逼真背景的人体动画方面达到领先性能。源代码与模型将发布于 https://github.com/liuxiaoyu1104/AnimateAnywhere。
原文摘要 · Abstract (English)
Human image animation aims to generate human videos of given characters and backgrounds that adhere to the desired pose sequence. However, existing methods focus more on human actions while neglecting the generation of background, which typically leads to static results or inharmonious movements. The community has explored camera pose-guided animation tasks, yet preparing the camera trajectory is impractical for most entertainment applications and ordinary users. As a remedy, we present an AnimateAnywhere framework, rousing the background in human image animation without requirements on camera trajectories. In particular, based on our key insight that the movement of the human body often reflects the motion of the background, we introduce a background motion learner (BML) to learn background motions from human pose sequences. To encourage the model to learn more accurate cross-frame correspondences, we further deploy an epipolar constraint on the 3D attention map. Specifically, the mask used to suppress geometrically unreasonable attention is carefully constructed by combining an epipolar mask and the current 3D attention map. Extensive experiments demonstrate that our AnimateAnywhere effectively learns the background motion from human pose sequences, achieving state-of-the-art performance in generating human animation results with vivid and realistic backgrounds. The source code and model will be available at https://github.com/liuxiaoyu1104/AnimateAnywhere.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。