让视频人物保持身份一致，同时生成自然动态的电影级画面。
MagicMirror: ID-Preserved Video Generation in Video Diffusion Transformers
- 用双分支提取人脸身份与结构特征，精准捕捉个体特征。
- 轻量级跨模态适配器实现高效身份融合，训练参数增量极小。
- 两阶段训练策略提升生成质量，适合影视、虚拟人等场景。
我们提出 MagicMirror，一种在视频扩散变换器基础上生成身份一致且具有电影级画质和动态运动的视频框架。尽管近期视频扩散模型在文本到视频生成方面取得显著进展，但在保持身份一致性的同时生成自然动作仍具挑战。以往方法要么需要针对个体微调，要么难以平衡身份保留与动作多样性。MagicMirror 引入三个关键组件：(1) 双分支面部特征提取器，用于捕捉身份与结构特征；(2) 基于条件自适应归一化的轻量级跨模态适配器，实现高效身份集成；(3) 结合合成身份对与视频数据的两阶段训练策略。大量实验表明，MagicMirror 在多个指标上优于现有方法，有效平衡身份一致性与自然运动，且新增参数极少。代码与模型将公开。
原文摘要 · Abstract (English)
We present MagicMirror, a framework for generating identity-preserved videos with cinematic-level quality and dynamic motion. While recent advances in video diffusion models have shown impressive capabilities in text-to-video generation, maintaining consistent identity while producing natural motion remains challenging. Previous methods either require person-specific fine-tuning or struggle to balance identity preservation with motion diversity. Built upon Video Diffusion Transformers, our method introduces three key components: (1) a dual-branch facial feature extractor that captures both identity and structural features, (2) a lightweight cross-modal adapter with Conditioned Adaptive Normalization for efficient identity integration, and (3) a two-stage training strategy combining synthetic identity pairs with video data. Extensive experiments demonstrate that MagicMirror effectively balances identity consistency with natural motion, outperforming existing methods across multiple metrics while requiring minimal parameters added. The code and model will be made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。