提出可解释的视觉变压器架构，让图像分类结果透明可读。
Disentangling Visual Transformers: Patch-level Interpretability for Image Classification
- 重新设计注意力机制，分离每个图像块的贡献
- 分类结果可看作图像块信息的线性组合
- 在可解释性与性能间取得良好平衡
视觉变换器在图像分类任务中表现卓越，但其性能提升以牺牲可解释性为代价。主要障碍在于自注意力机制会以复杂方式混合整幅图像的视觉信息。本文提出受视觉变换器启发的可解释架构Hindered Transformer（HiT），重新思考变换器设计，使分类阶段的图像块影响更易分离。最终，HiT 可被解释为图像块级信息的线性组合。实验表明，该方法在可解释性方面具有优势，且性能损失合理，适用于对可解释性要求高的应用场景。
原文摘要 · Abstract (English)
Visual transformers have achieved remarkable performance in image classification tasks, but this performance gain has come at the cost of interpretability. One of the main obstacles to the interpretation of transformers is the self-attention mechanism, which mixes visual information across the whole image in a complex way. In this paper, we propose Hindered Transformer (HiT), a novel interpretable by design architecture inspired by visual transformers. Our proposed architecture rethinks the design of transformers to better disentangle patch influences at the classification stage. Ultimately, HiT can be interpreted as a linear combination of patch-level information. We show that the advantages of our approach in terms of explicability come with a reasonable trade-off in performance, making it an attractive alternative for applications where interpretability is paramount.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。