让照片生成更灵活且保持身份特征，效果领先。
InfiniteYou: Flexible Photo Recrafting While Preserving Your Identity
- 通过残差连接注入身份特征，增强人物一致性。
- 多阶段训练提升图文对齐与图像质量，减少脸部复制痕迹。
- 可插拔设计兼容多种方法，适合内容创作与形象生成场景。
在先进扩散变压器（DiT）如FLUX背景下，实现灵活且高保真的人像身份保留生成仍具挑战。本文提出InfiniteYou（InfU），首个稳健利用DiT完成该任务的框架。InfU解决现有方法身份相似度不足、图文对齐差、生成质量低等问题。核心是InfuseNet，通过残差连接将身份特征注入DiT主模型，提升身份一致性和生成能力。采用多阶段训练策略，包括预训练和基于合成单人多样本（SPMS）数据的监督微调（SFT），进一步优化图文对齐、改善图像质量并缓解人脸复制现象。大量实验表明，InfU性能达到当前最优水平。此外，其即插即用设计可兼容多种已有方法，为社区提供重要贡献。
原文摘要 · Abstract (English)
Achieving flexible and high-fidelity identity-preserved image generation remains formidable, particularly with advanced Diffusion Transformers (DiTs) like FLUX. We introduce InfiniteYou (InfU), one of the earliest robust frameworks leveraging DiTs for this task. InfU addresses significant issues of existing methods, such as insufficient identity similarity, poor text-image alignment, and low generation quality and aesthetics. Central to InfU is InfuseNet, a component that injects identity features into the DiT base model via residual connections, enhancing identity similarity while maintaining generation capabilities. A multi-stage training strategy, including pretraining and supervised fine-tuning (SFT) with synthetic single-person-multiple-sample (SPMS) data, further improves text-image alignment, ameliorates image quality, and alleviates face copy-pasting. Extensive experiments demonstrate that InfU achieves state-of-the-art performance, surpassing existing baselines. In addition, the plug-and-play design of InfU ensures compatibility with various existing methods, offering a valuable contribution to the broader community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。