用可变形原型提升视觉Transformer的可解释性分类
Interpretable Image Classification with Adaptive Prototype-based Vision Transformers
- 基于ViT的可变形原型，支持不规则形变匹配图像
- 在多个数据集上优于现有原型模型，准确率更高
- 适合需要透明决策过程的医疗、安防等场景
我们提出ProtoViT，一种结合深度学习与案例推理的可解释图像分类方法。该方法通过将输入图像与一组可学习的原型进行比较来完成分类，提供‘这看起来像那’式的解释。在我们的模型中，原型由可变形的‘部分’构成，能适应物体的不规则几何变化，实现更精准的图像对比。不同于依赖卷积神经网络和空间刚性原型的现有模型，本工作首次将视觉变换器（ViT）引入原型模型，并设计可自适应调整数量的变形原型，既捕捉几何变化，又保持原型特征的连贯性和清晰性。实验表明，该模型在多个基准上均优于现有原型模型。全面分析验证了原型的一致性及解释的忠实性。
原文摘要 · Abstract (English)
We present ProtoViT, a method for interpretable image classification combining deep learning and case-based reasoning. This method classifies an image by comparing it to a set of learned prototypes, providing explanations of the form ``this looks like that.'' In our model, a prototype consists of \textit{parts}, which can deform over irregular geometries to create a better comparison between images. Unlike existing models that rely on Convolutional Neural Network (CNN) backbones and spatially rigid prototypes, our model integrates Vision Transformer (ViT) backbones into prototype based models, while offering spatially deformed prototypes that not only accommodate geometric variations of objects but also provide coherent and clear prototypical feature representations with an adaptive number of prototypical parts. Our experiments show that our model can generally achieve higher performance than the existing prototype based models. Our comprehensive analyses ensure that the prototypes are consistent and the interpretations are faithful.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。