开源工具包Prisma助力视觉模型可解释性研究
Prisma: An Open Source Toolkit for Mechanistic Interpretability in Vision and Video
- 统一框架支持75+视觉与视频Transformer模型
- 提供80+预训练稀疏自编码器权重，部分重构可降损
- 适合初学者与研究者快速入门视觉机制解析
稳健的工具链和公开预训练模型推动了语言模型机制可解释性的进展。但视觉模型的类似研究因缺乏易用框架和预训练权重而受阻。我们提出Prisma（代码库：https://github.com/Prisma-Multimodal/ViT-Prisma），一个开源框架，旨在加速视觉机制可解释性研究。该框架提供统一工具集，支持75+视觉与视频变压器模型，兼容稀疏自编码器（SAE）、编解码器和跨编码器训练；包含80+预训练SAE权重；具备激活缓存、电路分析与可视化工具，并配有教育资源。分析发现，有效的视觉SAE可呈现远低于语言模型的稀疏性模式，且在某些情况下，SAE重建甚至能降低模型损失。Prisma为理解视觉模型内部机制开辟新方向，同时降低该新兴领域入门门槛。
原文摘要 · Abstract (English)
Robust tooling and publicly available pre-trained models have helped drive recent advances in mechanistic interpretability for language models. However, similar progress in vision mechanistic interpretability has been hindered by the lack of accessible frameworks and pre-trained weights. We present Prisma (Access the codebase here: https://github.com/Prisma-Multimodal/ViT-Prisma), an open-source framework designed to accelerate vision mechanistic interpretability research, providing a unified toolkit for accessing 75+ vision and video transformers; support for sparse autoencoder (SAE), transcoder, and crosscoder training; a suite of 80+ pre-trained SAE weights; activation caching, circuit analysis tools, and visualization tools; and educational resources. Our analysis reveals surprising findings, including that effective vision SAEs can exhibit substantially lower sparsity patterns than language SAEs, and that in some instances, SAE reconstructions can decrease model loss. Prisma enables new research directions for understanding vision model internals while lowering barriers to entry in this emerging field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。