无需训练即可让任意角色动起来，支持各种风格人脸动画
FaceShot: Bring Any Character into Life
- 通过外观引导的地标匹配与坐标重映射，精准生成面部运动序列
- 在新构建的CharacBench数据集上超越现有最先进方法
- 兼容任何地标驱动动画模型，适合快速部署各类角色动画
本文提出FaceShot，一种无需训练的肖像动画框架，可将任意角色从任意驱动视频中“唤醒”而无需微调或重新训练。该方法通过外观引导的地标匹配模块和基于坐标的地标重映射模块，生成精确且鲁棒的重定位地标序列，利用潜在扩散模型的强语义对应关系，在多种角色类型下生成面部运动序列。随后将地标序列输入预训练的地标驱动动画模型，生成动画视频。凭借强大的泛化能力，FaceShot突破了真实肖像地标检测对风格化角色和驱动视频的限制，兼容任意地标驱动动画模型，显著提升整体性能。在新构建的角色基准数据集CharacBench上的大量实验表明，FaceShot在任一角色领域均持续超越现有最先进方法。
原文摘要 · Abstract (English)
In this paper, we present FaceShot, a novel training-free portrait animation framework designed to bring any character into life from any driven video without fine-tuning or retraining. We achieve this by offering precise and robust reposed landmark sequences from an appearance-guided landmark matching module and a coordinate-based landmark retargeting module. Together, these components harness the robust semantic correspondences of latent diffusion models to produce facial motion sequence across a wide range of character types. After that, we input the landmark sequences into a pre-trained landmark-driven animation model to generate animated video. With this powerful generalization capability, FaceShot can significantly extend the application of portrait animation by breaking the limitation of realistic portrait landmark detection for any stylized character and driven video. Also, FaceShot is compatible with any landmark-driven animation model, significantly improving overall performance. Extensive experiments on our newly constructed character benchmark CharacBench confirm that FaceShot consistently surpasses state-of-the-art (SOTA) approaches across any character domain. More results are available at our project website https://faceshot2024.github.io/faceshot/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。