用文本嵌入实现扩散模型的细粒度控制，免加载适配器且更快更省空间。
Prompt Sliders for Fine-Grained Control, Editing and Erasing of Concepts in Diffusion Models
- 通过文本嵌入学习概念，无需加载卸载LoRA适配器。
- 单个概念仅需3KB存储，比LoRA节省99.7%空间。
- 支持图像属性精细编辑与不良内容擦除，通用性强。
扩散模型在图像生成与编辑方面已超越GAN，但对生成图像属性的精确控制仍具挑战。现有概念滑块方法通过低秩适配器（LoRAs）学习概念，但需额外参数且推理慢，且适配器依赖具体模型架构，需为不同版本如Stable Diffusion v1.5和SD-XL重新训练。本文提出Prompt Sliders，一种基于文本反演的简单方法，通过文本嵌入学习概念，可在共享相同文本编码器的不同模型间通用。该方法不引入额外参数，仅需存储目标概念的文本嵌入，每个概念仅占3KB，远低于LoRA的8922KB以上。实验表明，该方法比使用LoRA快30%，同时支持新概念学习与不良概念（如艺术风格或成人内容）的擦除。
原文摘要 · Abstract (English)
Diffusion models have recently surpassed GANs in image synthesis and editing, offering superior image quality and diversity. However, achieving precise control over attributes in generated images remains a challenge. Concept Sliders introduced a method for fine-grained image control and editing by learning concepts (attributes/objects). However, this approach adds parameters and increases inference time due to the loading and unloading of Low-Rank Adapters (LoRAs) used for learning concepts. These adapters are model-specific and require retraining for different architectures, such as Stable Diffusion (SD) v1.5 and SD-XL. In this paper, we propose a straightforward textual inversion method to learn concepts through text embeddings, which are generalizable across models that share the same text encoder, including different versions of the SD model. We refer to our method as Prompt Sliders. Besides learning new concepts, we also show that Prompt Sliders can be used to erase undesirable concepts such as artistic styles or mature content. Our method is 30% faster than using LoRAs because it eliminates the need to load and unload adapters and introduces no additional parameters aside from the target concept text embedding. Each concept embedding only requires 3KB of storage compared to the 8922KB or more required for each LoRA adapter, making our approach more computationally efficient. Project Page: https://deepaksridhar.github.io/promptsliders.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。