arXiv:2604.10023cs.CVcs.AI2026-04

动态切换LoRA权重,实现高效风格迁移。

FREE-Switch: Frequency-based Dynamic LoRA Switch for Style Transfer

  • 根据频域重要性动态选择不同LoRA权重。
  • 在多个扩散步骤中保持语义一致,减少细节损失。
  • 无需训练,适合边缘设备部署,提升生成质量。

随着开源适配器在相同扩散模型基础上训练出多样场景与物体的生成能力,结合这些预训练权重可实现低成本定制化生成。然而,现有多数模型融合方法针对分类或文本生成设计,在图像生成中因多步扩散误差累积导致内容漂移。基于训练的方法计算成本高,不适合边缘部署;无训练方法采用统一融合策略,忽略适配器间差异,造成细节退化。我们发现不同适配器针对特定内容生成,各扩散步骤对每个适配器的重要性不同。因此提出基于频域重要性的动态LoRA切换方法。同时观察到保持适配器间语义一致性能有效缓解细节丢失,故设计自动生成对齐机制,在语义层面对齐生成意图。实验表明,FREE-Switch框架能高效组合不同对象与风格的适配器,显著降低高质量定制生成的训练成本。

原文摘要 · Abstract (English)

With the growing availability of open-sourced adapters trained on the same diffusion backbone for diverse scenes and objects, combining these pretrained weights enables low-cost customized generation. However, most existing model merging methods are designed for classification or text generation, and when applied to image generation, they suffer from content drift due to error accumulation across multiple diffusion steps. For image-oriented methods, training-based approaches are computationally expensive and unsuitable for edge deployment, while training-free ones use uniform fusion strategies that ignore inter-adapter differences, leading to detail degradation. We find that since different adapters are specialized for generating different types of content, the contribution of each diffusion step carries different significance for each adapter. Accordingly, we propose a frequency-domain importance-driven dynamic LoRA switch method. Furthermore, we observe that maintaining semantic consistency across adapters effectively mitigates detail loss; thus, we design an automatic Generation Alignment mechanism to align generation intents at the semantic level. Experiments demonstrate that our FREE-Switch (Frequency-based Efficient and Dynamic LoRA Switch) framework efficiently combines adapters for different objects and styles, substantially reducing the training cost of high-quality customized generation.

风格迁移LoRA扩散模型动态切换

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。