用3D几何流实现更精准的人像动画迁移与编辑。
FG-Portrait: 3D Flow Guided Editable Portrait Animation
- 基于3D头模直接计算运动对应关系，无需学习。
- 在保持原人物身份的同时，精准传递驱动动作。
- 支持表情和头部姿态的可控编辑，适合影视创作。
人像动画中的动作迁移仍是关键挑战。现有扩散模型仅依赖驱动动作，无法捕捉源图与驱动图间的对应关系，导致迁移效果不佳。虽然光流可提供替代方案，但2D输入下的稠密对应预测存在病态问题，常导致动画失真。本文提出3D流，一种无须训练、基于3D头模几何结构的运动对应关系计算方法。为将3D先验融入扩散模型，我们设计3D流编码机制，对每个目标像素查询其可能的3D位移以回溯至源位置。为进一步对齐2D动作变化，提出深度引导采样，精确定位每个像素对应的3D点。实验表明，该方法在保持源身份一致性的同时，显著提升动作迁移的一致性。此外，模型支持用户指定的表情与头姿编辑，具备更高可控性。
原文摘要 · Abstract (English)
Motion transfer from the driving to the source portrait remains a key challenge in the portrait animation. Current diffusion-based approaches condition only on the driving motion, which fails to capture source-to-driving correspondences and consequently yields suboptimal motion transfer. Although flow estimation provides an alternative, predicting dense correspondences from 2D input is ill-posed and often yields inaccurate animation. We address this problem by introducing 3D flows, a learning-free and geometry-driven motion correspondence directly computed from parametric 3D head models. To integrate this 3D prior into diffusion model, we introduce 3D flow encoding to query potential 3D flows for each target pixel to indicate its displacement back to the source location. To obtain 3D flows aligned with 2D motion changes, we further propose depth-guided sampling to accurately locate the corresponding 3D points for each pixel. Beyond high-fidelity portrait animation, our model further supports user-specified editing of facial expression and head pose. Extensive experiments demonstrate the superiority of our method on consistent driving motion transfer as well as faithful source identity preservation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。