用稀疏自编码器揭示CLIP视觉模型内部机制并实现精准控制
Steering CLIP's vision transformer with sparse autoencoders
- 在CLIP视觉变压器中训练稀疏自编码器，发现视觉与语言处理的差异
- 10%-15%神经元可被精确调控，自编码器提供数千个可调控特征
- 在中间层抑制特定特征可提升图像解耦任务性能，尤其在对抗攻击防御上领先
尽管视觉模型能力强大，其内部机制仍不清晰。稀疏自编码器（SAEs）在语言领域已有所突破，但在视觉领域尚属空白。本文首次在CLIP的视觉变压器上训练SAEs，揭示了跨层和不同标记类型下显著不同的稀疏模式。我们提出量化指标，系统分析模型可调控性，发现10%-15%的神经元和特征具有可调控性，且SAEs提供的可调控特征远超基础模型。通过有针对性地抑制SAE特征，我们在三个视觉解耦任务（CelebA、Waterbirds和字形攻击）中均取得改进，最优解耦效果出现在模型中间层，并在防御字形攻击任务上达到当前最佳表现。
原文摘要 · Abstract (English)
While vision models are highly capable, their internal mechanisms remain poorly understood -- a challenge which sparse autoencoders (SAEs) have helped address in language, but which remains underexplored in vision. We address this gap by training SAEs on CLIP's vision transformer and uncover key differences between vision and language processing, including distinct sparsity patterns for SAEs trained across layers and token types. We then provide the first systematic analysis on the steerability of CLIP's vision transformer by introducing metrics to quantify how precisely SAE features can be steered to affect the model's output. We find that 10-15\% of neurons and features are steerable, with SAEs providing thousands more steerable features than the base model. Through targeted suppression of SAE features, we then demonstrate improved performance on three vision disentanglement tasks (CelebA, Waterbirds, and typographic attacks), finding optimal disentanglement in middle model layers, and achieving state-of-the-art performance on defense against typographic attacks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。