arXiv:2504.15958cs.CV2025-04被引 2

无需训练,精准迁移主体身份,实现图文一致的图像生成。

FreeGraftor: Training-Free Cross-Image Feature Grafting for Subject-Driven Text-to-Image Generation

  • 通过跨图像特征嫁接,直接融合参考图细节
  • 零训练下主体保真度超越现有方法,文本对齐性强
  • 支持多主体生成,适合实际应用部署

主体驱动的图像生成旨在合成符合文本描述且忠实保留参考图像中主体身份的新场景。现有方法在保真度与效率间存在矛盾:微调类方法需耗时耗资源的专属优化,零样本方法则常难以维持足够主体一致性。本文提出FreeGraftor,一种无需训练的框架,通过跨图像特征嫁接解决上述问题。具体地,该框架利用语义匹配与位置约束注意力融合,将参考主体的视觉细节迁移至生成图像;同时引入新型噪声初始化策略,保留参考主体的几何先验,提升特征匹配鲁棒性。大量定性和定量实验表明,本方法在不进行模型微调或额外训练的前提下,显著优于现有零样本与无训练方法,在主体保真度与文本对齐性方面均表现优异。此外,该框架可无缝扩展至多主体生成,具备实际部署可行性。代码已开源:https://github.com/Nihukat/FreeGraftor。

原文摘要 · Abstract (English)

Subject-driven image generation aims to synthesize novel scenes that faithfully preserve subject identity from reference images while adhering to textual guidance. However, existing methods struggle with a critical trade-off between fidelity and efficiency. Tuning-based approaches rely on time-consuming and resource-intensive, subject-specific optimization, while zero-shot methods often fail to maintain adequate subject consistency. In this work, we propose FreeGraftor, a training-free framework that addresses these limitations through cross-image feature grafting. Specifically, FreeGraftor leverages semantic matching and position-constrained attention fusion to transfer visual details from reference subjects to the generated images. Additionally, our framework introduces a novel noise initialization strategy to preserve the geometry priors of reference subjects, facilitating robust feature matching. Extensive qualitative and quantitative experiments demonstrate that our method enables precise subject identity transfer while maintaining text-aligned scene synthesis. Without requiring model fine-tuning or additional training, FreeGraftor significantly outperforms existing zero-shot and training-free approaches in both subject fidelity and text alignment. Furthermore, our framework can seamlessly extend to multi-subject generation, making it practical for real-world deployment. Our code is available at https://github.com/Nihukat/FreeGraftor.

图像生成零样本主体保持无训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。