arXiv:2606.31699cs.CVcs.AI2026-06中稿 · ECCV

用稀疏自编码器检测目标区域,替换嵌入实现更干净的物体删除。

Look But Don't Touch with Sparse Autoencoders for Unlearning in Diffusion Models

论文配图:Look But Don't Touch with Sparse Autoencoders for Unlearning in Diffusion Models
图 1 · 摘自论文原文
  • 仅用SAE检测目标位置,不直接操控潜空间。
  • 替换含目标的图像块嵌入,避免分布外激活和视觉失真。
  • 揭示了概念检测与干预间的根本差距,适合做可解释性分析。

稀疏自编码器(SAEs)被视作可解释的概念级操作工具,假设孤立特征可作为可控干预点。本文系统评估了其在扩散模型中物体擦除与定向生成中的有效性。结果显示,尽管SAEs能可靠检测并定位扩散模型激活中的语义概念,但直接在潜空间干预常引发分布外激活,导致严重视觉瑕疵。为此,我们仅将SAE激活用作语义检测器,识别目标物体所在图像区域,并用不含该物体的块嵌入进行替换。该检测-替换方法保持了扩散模型的激活统计特性,显著优于潜空间导向。研究揭示:单义或稀疏特征并非天然适合作为控制开关用于定向。这表明SAE是强大的可解释性工具,但在直接操作如遗忘学习中存在重要局限。

原文摘要 · Abstract (English)

Sparse autoencoders (SAEs) have recently been proposed as interpretable tools for concept-level manipulation, under the assumption that isolated features can serve as controllable intervention points. In this work, we systematically evaluate this assumption in the context of object erasure and steering in diffusion models. We show that while SAEs reliably detect and localize semantic concepts within diffusion model activations, direct intervention in their latent space frequently induces out-of-distribution activations, resulting in severe visual artifacts. To disentangle detection from intervention, we use SAE activations purely as semantic detectors to identify image regions containing the target object, and replace those patch embeddings with the ones that do not contain it. This detection-based replacement preserves the diffusion model's activation statistics and produces significantly cleaner erasure results than latent steering. Our findings reveal a fundamental gap between concept detection and concept intervention in diffusion models: monosemantic or sparse features are not inherently suitable as control knobs for steering. These results position SAEs as powerful interpretability tools for analyzing generative models, but highlight important limitations when used for direct manipulation, such as unlearning.

扩散模型可解释性去偏自编码器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。