用一个LoRA整合50种图像特效，解决多特效部署时的干扰问题。
CollectionLoRA: Collecting 50 Effects in 1 LoRA via Multi-Teacher On-Policy Distillation
- 通过多教师在线蒸馏，将50个特效LoRA融合到单一模型中。
- 在保持特效精度的同时，部署开销大幅降低，生成速度更快。
- 适合需要高效集成多种视觉效果的生成应用开发者。
定制化图像编辑旨在使用少量成对数据，为预训练扩散模型添加特定视觉效果，通常通过低秩适配（LoRA）实现。随着所需效果数量增加，存储并动态加载多个特效LoRA会显著提升部署开销。此外，现有流程通常将多个特效LoRA与加速模块级联，引发严重参数干扰，导致概念泄露和风格退化。我们提出CollectionLoRA，一种多教师在线蒸馏框架，可将多达50种不同特效LoRA及其少步生成能力蒸馏至单一LoRA中，从根本上解决特征干扰问题并显著降低部署成本。具体方法包括：(i) 概率双流路由机制，在训练中随机切换数据源，增强模型在未见场景下的泛化能力；(ii) 非对称正交提示策略，实现在提示空间中的概念隔离；(iii) 粗到精蒸馏目标，缓解教师与学生模型间的分布差异。大量评估表明，CollectionLoRA成功将所有定制化特效及少步生成能力整合至单个LoRA中，降低部署开销，同时在概念保真度上达到或优于独立训练的教师模型。代码：https://github.com/Qwen-Applications/CollectionLoRA
原文摘要 · Abstract (English)
Customized image editing aims to equip pre-trained diffusion models with specific visual effects using limited paired data, typically via Low-Rank Adaptation (LoRA). As the number of desired effects grows, storing and dynamically loading numerous these effect LoRAs significantly increases deployment overhead. Furthermore, current pipelines typically cascade these effect LoRAs with acceleration modules for fast generation, which triggers severe parameter interference and results in concept bleeding and style degradation. We propose CollectionLoRA, a multi-teacher on-policy distillation framework capable of distilling the concepts of up to 50 different effect LoRAs along with few-step generation capabilities into a single LoRA. This fundamentally resolves the feature interference issue and significantly reduces deployment costs. Specifically, the method introduces (i) a Probabilistic Dual-Stream Routing mechanism that enables the model to randomly switch between data sources during training, effectively enhancing its generalization in unseen scenarios; (ii) an Asymmetric Orthogonal Prompting strategy to achieve concept isolation within the prompt space; (iii) a Coarse-to-Fine Distillation Objective to mitigate the distribution gap between the teacher and student models. Extensive evaluations show that CollectionLoRA distills all customized effects and few-step generation into a single LoRA, reducing deployment overhead while achieving concept fidelity comparable to or better than independently trained teacher models. Code: https://github.com/Qwen-Applications/CollectionLoRA
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。