发现扩散模型微调会悄悄破坏无关概念,提出新工具检测隐性漂移。
DriftScope: Measuring The Hidden Effects of Diffusion Model Adaptation

- 用软提示定位微调前后语义漂移的关键词
- 零样本分类显示部分类别准确率下降18.9点
- 适合关注模型稳定性与安全性的研究者
微调预训练文生图扩散模型时,现有评估仅关注目标效果,忽略潜在副作用。通过稀疏自编码器分析和零样本分类,我们发现微调会系统性损害语义无关概念,而聚合指标(如FID、KID)无法察觉:当指标响应时,模型已接近不可用;当模型仍可用时,指标保持平稳,但特定类别零样本准确率最高下降18.9点,概念分布发生显著偏移。该现象在概念定制与去学习两端均出现,表明是权重修改的系统性后果。为此,我们提出DriftScope,一种无需访问真实数据或模型内部结构的提示级诊断工具,可对任意两个模型检查点生成按概念漂移程度排序的词表,实现可解释的细粒度审计。
原文摘要 · Abstract (English)
Adapting pre-trained text-to-image diffusion models, whether to learn new visual concepts or erase unwanted ones, is routinely evaluated on its intended effects alone. We argue this framing is incomplete. Through sparse autoencoder analysis and zero-shot classification, we demonstrate that adaptation systematically damages semantically unrelated concepts in ways that aggregate metrics structurally cannot surface: when damage is severe enough for FID and KID to respond, the model is already nearly unusable; when the model remains functional, FID and KID stay flat while specific classes silently suffer worst-case zero-shot accuracy drops of up to 18.9 points and concept-level distributions shift dramatically. This pattern appears at both ends of the adaptation spectrum (concept customization and concept unlearning), suggesting it is a systematic consequence of weight-level modification rather than an artifact of any particular method. To surface this hidden drift before deployment, we introduce DriftScope, a prompt-level diagnostic tool that takes any two model checkpoints and returns a ranked list of tokens whose visual concepts have shifted most between them. DriftScope optimizes a soft prompt to attribute drift at the token level without requiring access to real data or model internals. The result is an interpretable, concept-level audit that aggregate evaluation cannot provide.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。