arXiv:2510.23515cs.CV2025-10

无需训练即可融合多个主体LoRA,实现精准图像生成。

FreeFuse: Multi-Subject LoRA Fusion via Adaptive Token-Level Routing at Test Time

  • 测试时通过自适应令牌级路由,动态分配LoRA残差到对应语义区域。
  • 在多主体生成中保持身份一致性和组合准确性,优于现有方法。
  • 无需额外训练或修改模型,用户只需提供主体关键词即可使用。

本文提出FreeFuse,一种无需训练的多主体文本到图像生成框架,通过自动融合多个主体LoRA实现。不同于以往需重训练以缓解特征冲突的方法,我们发现将LoRA残差空间路由至目标语义区域,能有效抑制跨区域干扰,同时保留基础模型的全局上下文推理能力。为此,我们在推理阶段引入自适应令牌级路由机制。为解决可靠路由区域获取难题,现有依赖文本-图像潜在关联的方法(如原始交叉注意力或概念级相似匹配)在处理视觉相似主体时常出现激活稀疏、空洞伪影和定位不稳问题,导致主体掩码不完整或模糊。为此,我们提出FreeFuseAttn,利用流匹配模型固有的语义对齐特性,在早期去噪步骤中动态匹配主体特定令牌与对应空间区域,从而避免对外部分割器的依赖。FreeFuse具有高度实用性:无需额外训练、模型修改或用户定义的空间约束。用户仅需提供主体激活词,即可无缝融入标准工作流程。大量实验验证,FreeFuse在身份保持和组合保真度方面均优于现有方法。

原文摘要 · Abstract (English)

This paper proposes FreeFuse, a training-free framework for multi-subject text-to-image generation through automatic fusion of multiple subject LoRAs. In contrast to prior studies that focus on retraining LoRAs to alleviate feature conflicts, our analysis shows that spatially routing LoRA residuals to their intended semantic regions provides an effective mechanism for suppressing direct cross-region LoRA interference while preserving the base model's global contextual reasoning. Accordingly, we implement Adaptive Token-Level Routing during the inference phase. However, obtaining reliable routing regions remains challenging. Existing methods that rely on text-image latent association, such as raw cross-attention or concept-level similarity matching, often suffer from sparse activations, hole artifacts, and unstable localization when handling visually similar subjects, leading to incomplete or ambiguous subject masks. To address these issues, we introduce FreeFuseAttn, a mechanism that exploits the flow matching model's intrinsic semantic alignment to dynamically match subject-specific tokens to their corresponding spatial regions at early denoising timesteps, thereby bypassing the need for external segmentors. FreeFuse distinguishes itself through high practicality: it necessitates no additional training, model modifications, or user-defined spatial constraints. Users need only provide subject activation words to achieve seamless integration into standard workflows. Extensive experiments validate that FreeFuse outperforms existing approaches in both identity preservation and compositional fidelity. The project page is at: https://future-item.github.io/FreeFuse/.

LoRA融合文本生成图像生成无训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。