arXiv:2604.07166cs.CVcs.HC2026-04中稿 · the 5th Explainabl…

让复杂模型的图像分类结果可解释,且精度更高

DINO-QPM: Adapting Visual Foundation Models for Globally Interpretable Image Classification

  • 用平均池化替代分类令牌,实现空间定位的可解释特征
  • 通过稀疏性损失减少背景干扰,提升解释聚焦度
  • 适配冻结的DINO模型,兼顾高精度与人类可读性

尽管DINOv2等视觉基础模型作为特征提取器表现优异,但其复杂高维表示严重阻碍可解释性。本文提出DINO-QPM,将这些强大但纠缠的特征转化为对比式、类无关的可解释表示。该方法是轻量级可解释性适配器,基于二次规划增强模型(QPM)在严格冻结的DINO主干上运行。不同于通常依赖 exttt{CLS}标记的分类方式,我们采用平均池化,直接连接补丁嵌入与模型特征,从而在输入空间中实现全局可解释特征的空间定位。此外,引入稀疏性损失以最小化空间散射和背景噪声,确保解释聚焦于相关物体部分。实验表明,DINO-QPM在保持高分类准确率的同时,使QPM的可解释水平得以适配,优于DINOv2线性探测器,并在新提出的合理性指标及其他可解释性指标上显著领先于其他适用于冻结视觉基础模型的方法。

原文摘要 · Abstract (English)

Although visual foundation models like DINOv2 provide state-of-the-art performance as feature extractors, their complex, high-dimensional representations create substantial hurdles for interpretability. This work proposes DINO-QPM, which converts these powerful but entangled features into contrastive, class-independent representations that are interpretable by humans. DINO-QPM is a lightweight interpretability adapter that pursues globally interpretable image classification, adapting the Quadratic Programming Enhanced Model (QPM) to operate on strictly frozen DINO backbones. While classification with visual foundation models typically relies on the \texttt{CLS} token, we deliberately diverge from this standard. By leveraging average-pooling, we directly connect the patch embeddings to the model's features and therefore enable spatial localisation of DINO-QPM's globally interpretable features within the input space. Furthermore, we apply a sparsity loss to minimise spatial scatter and background noise, ensuring that explanations are grounded in relevant object parts. With DINO-QPM we make the level of interpretability of QPM available as an adapter while exceeding the accuracy of DINOv2 linear probe. Evaluated through an introduced Plausibility metric and other interpretability metrics, extensive experiments demonstrate that DINO-QPM is superior to other applicable methods for frozen visual foundation models in both classification accuracy and explanation quality.

可解释性视觉模型DINOv2特征定位

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。