用稀疏自编码器揭示视觉模型特征语义并实现可控实验验证。
Interpretable and Testable Vision Features via Sparse Autoencoders
- 通过稀疏自编码器提取可解释的视觉特征，附带真实图像示例和可操控的解码向量。
- 单个SAE在冻结ViT上实现分类与分割任务的像素级因果编辑，无需重训练。
- 适合关注模型可解释性与因果推理的研究者或开发者。
为真正理解视觉模型，不仅需要解释其学习到的特征，还需通过受控实验验证这些解释。现有方法通常只提供丰富语义或直接控制中的一种，缺乏兼具二者的通用后处理工具。本文采用稀疏自编码器（SAEs），每个稀疏特征均配有真实图像样例以揭示其语义，并附有可操纵的解码向量来探测其对下游任务行为的影响。在多个预训练视觉模型上应用该方法，揭示了不同预训练目标所学习语义抽象的显著差异。进一步表明，仅用一个在冻结ViT激活上训练的SAE，即可在分类与分割任务中实现无须重训练ViT或任务头的像素级因果编辑。这些定性且可证伪的演示确立了SAEs作为概念发现与因果探查之间实用桥梁的地位。代码、演示与模型见项目网站：https://osu-nlp-group.github.io/saev。
原文摘要 · Abstract (English)
To truly understand vision models, we must not only interpret their learned features but also validate these interpretations through controlled experiments. While earlier work offers either rich semantics or direct control, few post-hoc tools supply both in a single, model-agnostic procedure. We use sparse autoencoders (SAEs) to bridge this gap; each sparse feature comes with real-image exemplars that reveal its meaning and a decoding vector that can be manipulated to probe its influence on downstream task behavior. By applying our method to widely-used pre-trained vision models, we reveal meaningful differences in the semantic abstractions learned by different pre-training objectives. We then show that a single SAE trained on frozen ViT activations supports patch-level causal edits across tasks (classification and segmentation) all without retraining the ViT or task heads. These qualitative, falsifiable demonstrations position SAEs as a practical bridge between concept discovery and causal probing of vision models. We provide code, demos and models on our project website: https://osu-nlp-group.github.io/saev.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。