MagicID通过偏好学习实现视频身份一致且动态自然的个性化生成。
MagicID: Hybrid Preference Optimization for ID-Consistent and Dynamic-Preserved Video Customization
- 构建带身份与动态奖励的成对视频数据,替代传统自重构训练。
- 在长视频中保持身份一致性,动态质量优于现有方法。
- 适合需要高保真个性化视频生成的用户和研究者。
视频身份定制旨在生成保持身份一致性且具有显著动态特性的高质量视频。然而,现有方法面临两个关键挑战:长时间视频中身份退化,以及训练过程中动态性降低,主要源于依赖静态图像的自重构训练。为此,我们提出魔力身份(MagicID)框架,直接促进生成符合用户偏好的身份一致且动态丰富的视频。具体地,我们构建包含显式身份与动态奖励的成对偏好视频数据,用于偏好学习,而非沿用传统自重构方式。为克服定制化偏好数据的局限,引入混合采样策略:首先利用参考图像生成的静态视频优先保障身份保留,再通过基于前沿的采样方法提升生成视频的动态运动质量。借助这些混合偏好对,优化模型以对齐不同定制偏好间的奖励差异。大量实验表明,MagicID成功实现了身份一致性和自然动态性,在多项指标上超越现有方法。
原文摘要 · Abstract (English)
Video identity customization seeks to produce high-fidelity videos that maintain consistent identity and exhibit significant dynamics based on users' reference images. However, existing approaches face two key challenges: identity degradation over extended video length and reduced dynamics during training, primarily due to their reliance on traditional self-reconstruction training with static images. To address these issues, we introduce $\textbf{MagicID}$, a novel framework designed to directly promote the generation of identity-consistent and dynamically rich videos tailored to user preferences. Specifically, we propose constructing pairwise preference video data with explicit identity and dynamic rewards for preference learning, instead of sticking to the traditional self-reconstruction. To address the constraints of customized preference data, we introduce a hybrid sampling strategy. This approach first prioritizes identity preservation by leveraging static videos derived from reference images, then enhances dynamic motion quality in the generated videos using a Frontier-based sampling method. By utilizing these hybrid preference pairs, we optimize the model to align with the reward differences between pairs of customized preferences. Extensive experiments show that MagicID successfully achieves consistent identity and natural dynamics, surpassing existing methods across various metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。