arXiv:2602.18936cs.CV2026-02被引 2

通过约束秩的适配与免训练融合,实现内容与风格解耦的个性化图像生成。

CRAFT-LoRA: Content-Style Personalization via Rank-Constrained Adaptation and Training-Free Fusion

  • 采用低秩投影残差注入,分离内容与风格子空间。
  • 支持无训练融合,跨扩散步骤动态调整噪声预测以提升稳定性。
  • 无需额外训练即可灵活控制多个LoRA模块的组合效果。

个性化图像生成需在文本和参考图像基础上平衡内容保真度与风格一致性。低秩适配(LoRA)提供了高效个性化的途径,通过组合不同概念的LoRA权重可实现精确控制。然而现有组合方法仍存在内容与风格表示纠缠、元素影响力控制不足、权重融合不稳定且常需额外训练等问题。本文提出CRAFT-LoRA,包含三个互补组件:(1) 基于秩约束的骨干微调,通过注入低秩投影残差,促进内容与风格子空间的解耦学习;(2) 基于提示引导的专家编码器,具备专用分支结构,实现语义扩展与选择性适配器聚合,支持精准控制;(3) 免训练的、时间步依赖的无分类器指导方案,通过在扩散过程中策略性调节噪声预测,增强生成稳定性。本方法显著提升内容-风格解耦能力,支持灵活语义控制下的LoRA模块组合,且无需额外训练开销即可实现高保真生成。

原文摘要 · Abstract (English)

Personalized image generation requires effectively balancing content fidelity with stylistic consistency when synthesizing images based on text and reference examples. Low-Rank Adaptation (LoRA) offers an efficient personalization approach, with potential for precise control through combining LoRA weights on different concepts. However, existing combination techniques face persistent challenges: entanglement between content and style representations, insufficient guidance for controlling elements' influence, and unstable weight fusion that often require additional training. We address these limitations through CRAFT-LoRA, with complementary components: (1) rank-constrained backbone fine-tuning that injects low-rank projection residuals to encourage learning decoupled content and style subspaces; (2) a prompt-guided approach featuring an expert encoder with specialized branches that enables semantic extension and precise control through selective adapter aggregation; and (3) a training-free, timestep-dependent classifier-free guidance scheme that enhances generation stability by strategically adjusting noise predictions across diffusion steps. Our method significantly improves content-style disentanglement, enables flexible semantic control over LoRA module combinations, and achieves high-fidelity generation without additional retraining overhead.

图像生成LoRA风格解耦扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。