用神经科学启发的机制解释ViT，让视觉模型决策可读可调。
Inside the Visual Mind: Neuroscience-Motivated Concept Circuits for Interpreting and Steering Vision Transformers
- 基于64K图像和16K概念库构建高效概念探测器，覆盖效率提升20倍。
- 自动识别概念电路，修复模型在WaterBirds数据集上最差组准确率48.2%。
- 适合研究模型可解释性与安全部署的开发者与研究人员使用。
尽管视觉Transformer（ViT）精度高，但其预测可能受无关线索干扰，亟需理解其内部机制以确保安全应用。稀疏自编码器（SAEs）为将模型表示分解为人类可理解的概念提供了前景，但将其应用于ViT仍面临概念覆盖率有限、特征解释主观且不可扩展的挑战。受神经科学启发，我们提出ViSAE——一种基于概念电路的机械可解释性工具箱，用于解析ViT内部运作。ViSAE包含三部分：(1) 一个包含64,000张图像和16,000个视觉锚定概念词汇的探测套件，相比ImageNet提升概念覆盖效率20倍，解释准确率提高28.7%；(2) 自顶向下概念读取与自底向上电路追踪算法，可自动还原ViT内部工作机制；(3) 用于审计与调控模型行为的应用。通过概念编辑，ViSAE在WaterBirds数据集上将最差组准确率提升48.2%，优于现有方法23.8%。数据与代码见：https://github.com/deep-real/ViSAE。
原文摘要 · Abstract (English)
Despite high accuracy, Vision Transformer (ViT) predictions can be driven by spurious cues, raising the need to understand their inner workings before safe deployment. Sparse autoencoders (SAEs) provide a promising lens for decomposing model representations into human-interpretable concepts, yet adapting SAE-based interpretation to ViTs remains challenging due to limited control over concept coverage and subjective, non-scalable feature interpretation. To fill the gaps, motivated by neuroscience-inspired principles, we propose ViSAE, a mechanistic interpretability toolbox for understanding ViT inner workings through concept circuits. ViSAE consists of three components: (1) A probing suite with 64K images and a 16K visually grounded concept vocabulary, improving concept coverage efficiency by 20x over ImageNet and interpretation accuracy by 28.7% over existing concept sets. (2) Top-down concept reading and Bottom-up circuit tracing algorithms that automatically recover ViT inner workings via concept circuits. (3) Applications for auditing and steering ViT behavior. Through concept editing, ViSAE improves the worst-group accuracy on WaterBirds by 48.2%, outperforming existing methods by 23.8%. Our data and code: https://github.com/deep-real/ViSAE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。