arXiv:2512.03964cs.CV2025-12

统一融合两种方法,实现高保真人脸个性化与灵活文本控制。

Training for Identity, Inference for Controllability: A Unified Approach to Tuning-Free Face Personalization

  • 训练时聚焦身份特征,双分支协同学习身份信息。
  • 推理时通过归一化重缩放,恢复扩散模型的文本控制能力。
  • 无需微调,适合需快速定制人脸的生成应用。

无微调的人脸个性化方法发展出两类范式:一类将面部特征映射到文本嵌入空间,另一类通过辅助交叉注意力层注入特征。尽管两者均有潜力,但现有方法难以同时实现高身份保真度和灵活的文本可控性。本文提出UniID,一种统一的无微调框架,协同整合两类范式。核心洞察是:融合时仅应强化与身份相关的信息,同时保留原始扩散先验以处理非身份属性。通过设计一致的训练-推理策略:训练阶段采用身份导向学习,引导双分支仅捕捉身份特征;推理阶段引入归一化重缩放机制,恢复基线扩散模型的文本可控性,并使互补的身份信号相互增强。该设计使UniID在保持高保真人脸个性化的同时具备灵活文本控制能力。在六种先进方法上的大量实验表明,UniID在身份保留和文本可控性上均表现更优。代码将发布于 https://github.com/lyuPang/UniID。

原文摘要 · Abstract (English)

Tuning-free face personalization methods have developed along two distinct paradigms: text embedding approaches that map facial features into the text embedding space, and adapter-based methods that inject features through auxiliary cross-attention layers. While both paradigms have shown promise, existing methods struggle to simultaneously achieve high identity fidelity and flexible text controllability. We introduce UniID, a unified tuning-free framework that synergistically integrates both paradigms. Our key insight is that when merging these approaches, they should mutually reinforce only identity-relevant information while preserving the original diffusion prior for non-identity attributes. We realize this through a principled training-inference strategy: during training, we employ an identity-focused learning scheme that guides both branches to capture identity features exclusively; at inference, we introduce a normalized rescaling mechanism that recovers the text controllability of the base diffusion model while enabling complementary identity signals to enhance each other. This principled design enables UniID to achieve high-fidelity face personalization with flexible text controllability. Extensive experiments against six state-of-the-art methods demonstrate that UniID achieves superior performance in both identity preservation and text controllability. Code will be available at https://github.com/lyuPang/UniID

人脸生成无微调扩散模型可控生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。