让多个角色共存于同一图像，不混淆身份也不乱了风格。
LILAC: Layer-Wise Independent LoRAs and Cascaded Conditioning for Multi-Concept Customization of Diffusion Models
- 用分层独立适配器，每次只激活一个角色的参数。
- 在Qwen模型上实现0.861的辨识率，效果稳定且可扩展。
- 适合需要多主题定制的图像生成场景，如角色创作、广告设计。
个性化文生图扩散模型以在统一图像中渲染多个特定主体仍具挑战：模型需保持每个主体的身份一致性，同时保证场景的空间与视觉连贯性。现有方法通过联邦平均、梯度融合或正交约束将独立训练的概念适配器融合于共享权重空间，常导致身份混淆和风格泄露。本文提出LILAC框架，采用分层独立低秩适配器（LoRAs）在推理时组合概念：每个主体基于先前已放置主体的冻结复合条件进行生成，且仅一个适配器激活，从而避免参数层面的干扰。该方法无需联合训练，随概念数量线性扩展，且对主干网络无依赖。在正交适配协议下，应用在Qwen-Image-Edit+Qwen-Image-Layered上实现0.861的ArcFace检测率。代码已开源。
原文摘要 · Abstract (English)
Personalizing text-to-image diffusion models to render several specific subjects in a coherent image remains challenging: the model must preserve each subject's identity while keeping the scene spatially and visually coherent. Methods that fuse independently trained concept adapters in a shared weight space (via federated averaging, gradient fusion, or orthogonality constraints) suffer from identity confusion and style bleeding. In this work, we show that composing concepts as separate image layers, instead of merging their adapters in a shared weight space, avoids parameter-level interference. We introduce LILAC, a framework that composes independently trained low-rank adapters at inference time: each subject is conditioned on the frozen composite of previously placed subjects, with exactly one adapter active at a time, therefore identities never interfere at the parameter level. LILAC composes the adapters without joint training, scales linearly with the number of concepts, and is backbone-agnostic. Under the Orthogonal Adaptation protocol, LILAC applied on Qwen-Image-Edit+Qwen-Image-Layered reaches an ArcFace detection rate of 0.861. Code is available at https://github.com/marianlupascu/LILAC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。