解决人物图像生成中面部与整体形象的协调难题
Beyond Facial Consistency: Personalized Person Image Generation with Holistic Identity Preservation

- 双分支框架统一控制面部与整体外观
- 动态平衡策略提升面部与整体一致性
- 新基准数据集支持全方位身份评估
个性化人物图像生成需同时保持局部面部细节与整体外观特征的一致性。现有方法通常只关注单一层面的身份信息,导致面部保真度与整体外观一致性之间存在固有权衡。为此,我们提出一个简单的双分支基线模型,将全局外观控制与局部面部控制整合于同一生成框架中。该组合虽取得良好效果,但在实际应用中因分支贡献不协调而表现不稳定。为此,我们提出动态平衡缩放(DBS)微调策略,包含自适应时间门控(沿去噪轨迹动态调节分支贡献)和区域感知优化(增强面部、外观与全局监督间的协同)。两项设计有效缓解了面部分支过主导问题,促进更有效的外观引导。我们还构建了Pexels-100数据集,用于评估个性化人物生成中的整体身份一致性。实验表明,DBS在面部保真度与外观一致性之间实现更优权衡,且提供可控的统一身份建模基础框架。
原文摘要 · Abstract (English)
Personalized person image generation requires preserving subject identity across both local facial details and broader appearance cues. Existing methods typically emphasize only one level of identity information, leading to an inherent trade-off between facial fidelity and overall appearance consistency. To address this, we first propose a simple dual-branch baseline that unifies global appearance control and local facial control within a shared generation framework. This simple combination of different branches yields promising results, but suffers from instability in practice due to uncoordinated branch contributions. To this end, we propose Dynamic Balancing Scaling (DBS), a fine-tuning strategy for improving face and appearance identity coordination. DBS consists of two components: adaptive temporal gating, which dynamically modulates branch contributions along the denoising trajectory, and region-aware optimization, which improves the coordination of facial, appearance, and global supervision. Together, these designs alleviate persistent face-branch over-dominance and encourage more effective appearance-aware guidance. We also introduce Pexels-100, a benchmark for evaluating holistic identity consistency in personalized person generation. Experiments show that DBS achieves a better trade-off between facial fidelity and appearance consistency than existing open-source baselines, while providing a controllable basic framework for holistic identity modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。