用视觉大模型+原型架构，让分类结果自动解释原因。
Tell me why: Visual foundation models as self-explainable classifiers
- 在冻结的视觉大模型上加轻量头，通过原型学习实现可解释分类。
- 分类性能媲美顶尖模型，解释性指标全面超越现有方法。
- 适合需要可信决策依据的医疗、金融等高风险场景。
视觉基础模型（VFMs）因卓越性能广受欢迎，但在关键应用中可解释性仍至关重要。自解释模型（SEM）旨在将预测分解为可理解概念的加权和，但现有方法的解释常缺乏真实性。本文结合VFMs与新型原型架构及专用训练目标，仅在冻结的VFMs上训练约100万参数的轻量级头部，提出ProtoFM。实验表明，该方法在保持竞争性分类性能的同时，在多项文献中提出的可解释性度量上均优于现有模型。代码已开源。
原文摘要 · Abstract (English)
Visual foundation models (VFMs) have become increasingly popular due to their state-of-the-art performance. However, interpretability remains crucial for critical applications. In this sense, self-explainable models (SEM) aim to provide interpretable classifiers that decompose predictions into a weighted sum of interpretable concepts. Despite their promise, recent studies have shown that these explanations often lack faithfulness. In this work, we combine VFMs with a novel prototypical architecture and specialized training objectives. By training only a lightweight head (approximately 1M parameters) on top of frozen VFMs, our approach (ProtoFM) offers an efficient and interpretable solution. Evaluations demonstrate that our approach achieves competitive classification performance while outperforming existing models across a range of interpretability metrics derived from the literature. Code is available at https://github.com/hturbe/proto-fm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。