无需训练即可融合任意主体与风格的LoRA，保留两者核心特征。
K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAs
- 通过比较各LoRA的前K个注意力元素，动态选择最优融合路径。
- 在多个数据集上超越现有训练型方法，在定性和定量上均表现更优。
- 适合需要快速组合不同风格与主体的生成任务用户使用。
近期研究尝试结合不同LoRA以联合生成学习到的风格与内容，但现有方法要么无法同时有效保留原始主体与风格，要么需要额外训练。本文认为LoRA的内在特性可有效引导扩散模型融合学习到的主体与风格。基于此，我们提出K-LoRA，一种简单且有效的免训练LoRA融合方法。在每一注意力层中，K-LoRA比较待融合的LoRA的前K个元素，决定哪个LoRA用于最优融合。该选择机制确保主体与风格最具代表性的特征在融合过程中得以保留，有效平衡二者贡献。实验结果表明，所提方法能有效整合原生LoRA学习到的主体与风格信息，在定性与定量结果上均优于当前最先进的训练型方法。
原文摘要 · Abstract (English)
Recent studies have explored combining different LoRAs to jointly generate learned style and content. However, existing methods either fail to effectively preserve both the original subject and style simultaneously or require additional training. In this paper, we argue that the intrinsic properties of LoRA can effectively guide diffusion models in merging learned subject and style. Building on this insight, we propose K-LoRA, a simple yet effective training-free LoRA fusion approach. In each attention layer, K-LoRA compares the Top-K elements in each LoRA to be fused, determining which LoRA to select for optimal fusion. This selection mechanism ensures that the most representative features of both subject and style are retained during the fusion process, effectively balancing their contributions. Experimental results demonstrate that the proposed method effectively integrates the subject and style information learned by the original LoRAs, outperforming state-of-the-art training-based approaches in both qualitative and quantitative results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。