通过稀疏自编码器实现文本嵌入的精准可控图像编辑
SAEdit: Token-level control for continuous image editing via Sparse AutoEncoder
- 用稀疏自编码器提取语义独立的文本嵌入方向
- 可连续调节编辑强度，实现属性间解耦控制
- 无需修改扩散模型，适配多种生成框架
大规模文生图扩散模型已成为现代图像编辑的核心，但仅靠文本提示难以实现精细控制。理想的编辑应具备解耦性（修改一项属性不干扰其他）和连续性（编辑强度可平滑调节）。本文提出一种基于文本嵌入的令牌级控制方法，通过在特定方向上操纵嵌入向量来实现对目标属性强度的精确控制。该方向由稀疏自编码器（SAE）的稀疏潜在空间识别，其隐变量具有语义隔离性。方法直接作用于文本嵌入，不改变扩散过程，因此与模型无关，适用于多种图像生成架构。实验表明，该方法可在不同属性和领域中实现直观、高效的连续编辑。
原文摘要 · Abstract (English)
Large-scale text-to-image diffusion models have become the backbone of modern image editing, yet text prompts alone do not offer adequate control over the editing process. Two properties are especially desirable: disentanglement, where changing one attribute does not unintentionally alter others, and continuous control, where the strength of an edit can be smoothly adjusted. We introduce a method for disentangled and continuous editing through token-level manipulation of text embeddings. The edits are applied by manipulating the embeddings along carefully chosen directions, which control the strength of the target attribute. To identify such directions, we employ a Sparse Autoencoder (SAE), whose sparse latent space exposes semantically isolated dimensions. Our method operates directly on text embeddings without modifying the diffusion process, making it model agnostic and broadly applicable to various image synthesis backbones. Experiments show that it enables intuitive and efficient manipulations with continuous control across diverse attributes and domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。