让一张图的内容和风格彻底分离,实现自由重用。
SplitFlux: Learning to Decouple Content and Style from a Single Image
- 通过LoRA微调Flux模型的单流块,分离图像内容与风格。
- 在多个数据集上内容保留率超90%,风格迁移质量显著提升。
- 适合需要精准控制图像内容的生成任务,如艺术创作、设计复用。
图像内容与风格的解耦对个性化图像生成至关重要。现有基于SDXL的方法难以达到高质量效果,而近期提出的Flux模型因未充分挖掘其特性,未能实现有效的内容-风格分离。为此,我们对Flux进行了系统分析,发现两个关键现象:(1) 单流块对图像生成至关重要;(2) 早期单流块主要控制内容,后期则主导风格。基于此,我们提出SplitFlux,通过LoRA微调单流块实现内容与风格解耦,使解耦后的内容可无缝嵌入新场景。其核心包括:(1) 低秩约束适应:压缩特定块的更新秩并放大幅度,防止内容泄露至风格块;(2) 视觉门控LoRA:将内容LoRA分为双分支,根据图像显著性分配不同秩,高秩分支保留主体信息,低秩分支编码残差细节,缓解过拟合。大量实验表明,SplitFlux在多种场景下持续优于当前最优方法,在内容保留与风格化质量上均表现卓越。
原文摘要 · Abstract (English)
Disentangling image content and style is essential for customized image generation. Existing SDXL-based methods struggle to achieve high-quality results, while the recently proposed Flux model fails to achieve effective content-style separation due to its underexplored characteristics. To address these challenges, we conduct a systematic analysis of Flux and make two key observations: (1) Single Stream Blocks are essential for image generation; and (2) Early single stream blocks mainly control content, whereas later blocks govern style. Based on these insights, we propose SplitFlux, which disentangles content and style by fine-tuning the single stream blocks via LoRA, enabling the disentangled content to be re-embedded into new contexts. It includes two key components: (1) Rank-Constrained Adaptation. To preserve content identity and structure, we compress the rank and amplify the magnitude of updates within specific blocks, preventing content leakage into style blocks. (2) Visual-Gated LoRA. We split the content LoRA into two branches with different ranks, guided by image saliency. The high-rank branch preserves primary subject information, while the low-rank branch encodes residual details, mitigating content overfitting and enabling seamless re-embedding. Extensive experiments demonstrate that SplitFlux consistently outperforms state-of-the-art methods, achieving superior content preservation and stylization quality across diverse scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。