直接从视频学动作，无需依赖易错的姿势估计。
Beyond Skeletons: Learning Animation Directly from Driving Videos with Same2X Training Strategy
- 跳过姿势提取，直接用原始视频驱动动画生成。
- 新设计三重线索融合，提升动作、表情和对齐控制力。
- 跨身份训练策略加速收敛，适合身份不一致场景。
人体图像动画旨在从静态参考图生成视频，由驱动视频中提取的姿态信息引导。现有方法常依赖姿态估计算法获取中间表示，但在遮挡或复杂姿态下易出错。本文提出DirectAnimator框架，绕过姿态提取，直接从原始驱动视频学习。引入包含姿态、面部和位置三重线索的驾驶提示，以语义丰富且稳定的形态捕捉运动、表情与对齐信息，并通过CueFusion DiT模块在去噪过程中实现可靠控制。为提升跨身份场景下的学习稳定性，设计Same2X训练策略，将跨身份特征对齐至同身份数据所学特征，正则化优化并加速收敛。大量实验表明，DirectAnimator在视觉质量与身份保留方面达到当前最优，且对遮挡和复杂动作具有鲁棒性，同时计算资源更少。项目页：https://directanimator.github.io/
原文摘要 · Abstract (English)
Human image animation aims to generate a video from a static reference image, guided by pose information extracted from a driving video. Existing approaches often rely on pose estimators to extract intermediate representations, but such signals are prone to errors under occlusion or complex poses. Building on these observations, we present DirectAnimator, a framework that bypasses pose extraction and directly learns from raw driving videos. We introduce a Driving Cue Triplet consisting of pose, face, and location cues that captures motion, expression, and alignment in a semantically rich yet stable form, and we fuse them through a CueFusion DiT block for reliable control during denoising. To make learning dependable when the driving and reference identities differ, we devise a Same2X training strategy that aligns cross-ID features with those learned from same-ID data, regularizing optimization and accelerating convergence. Extensive experiments demonstrate that DirectAnimator attains state-of-the-art visual quality and identity preservation while remaining robust to occlusions and complex articulation, and it does so with fewer computational resources. Our project page is at https://directanimator.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。