用可解释特征替换残差流，看清ViT如何逐步理解图像。
Interpreting vision transformers via residual replacement model
- 通过稀疏自编码器提取6600个特征，分析各层特征演变
- 发现低层特征处理纹理、高层特征捕捉语义，还有专门编码曲线和位置的特征
- 模型简化原计算流程，让人类能直观理解ViT运作机制
视觉变换器（ViTs）如何表征与处理世界？本文首次系统分析了通过稀疏自编码器提取的全部层级中6600个特征，并提出残差替换模型，将ViT计算替换为可解释特征。分析揭示：特征从低层模式逐步演变为高层语义，且存在专门编码曲线与空间位置的特征类型。该模型可规模化生成忠实而简洁的解释性电路，显著简化原始计算过程，实现人尺度上的可解释性。最终，本框架被用于消除虚假相关性，展现其实际应用价值。
原文摘要 · Abstract (English)
How do vision transformers (ViTs) represent and process the world? This paper addresses this long-standing question through the first systematic analysis of 6.6K features across all layers, extracted via sparse autoencoders, and by introducing the residual replacement model, which replaces ViT computations with interpretable features in the residual stream. Our analysis reveals not only a feature evolution from low-level patterns to high-level semantics, but also how ViTs encode curves and spatial positions through specialized feature types. The residual replacement model scalably produces a faithful yet parsimonious circuit for human-scale interpretability by significantly simplifying the original computations. As a result, this framework enables intuitive understanding of ViT mechanisms. Finally, we demonstrate the utility of our framework in debiasing spurious correlations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。