让视觉模型决策过程可解释，还能零样本完成新任务。
Multimodal Concept Bottleneck Models
- 用双瓶颈层对齐图像与文本特征，实现跨模态可解释性。
- 在四个基准上平均提升51.26%准确率，接近黑盒模型性能。
- 适合需要透明决策的场景，如医疗影像或安全敏感应用。
概念瓶颈模型（CBMs）通过将图像特征与自然概念对齐，提升深度学习模型的可解释性。然而现有方法受限于固定类别集，且存在非概念信息泄漏风险。本文提出多模态概念瓶颈模型（MM-CBM），将其扩展至CLIP框架。MM-CBM采用双概念瓶颈层（CBLs），分别对齐图像与文本嵌入，生成可解释特征。由此支持零样本分类、图像检索等新视觉任务。相较于现有方法，MM-CBM在四个标准基准上平均提升51.26%准确率，同时保持与黑盒模型相差约5%的精度，显著增强可解释性。
原文摘要 · Abstract (English)
Concept Bottleneck Models (CBMs) enhance the interpretability of deep learning networks by aligning the features extracted from images with natural concepts. However, existing CBMs are constrained in their ability to generalize beyond a fixed set of predefined classes and the risk of non-concept information leakage, where predictive signals outside the intended concepts are inadvertently exploited. In this paper, we propose Multimodal Concept Bottleneck Model (MM-CBM) to address these issues and extend CBMs into CLIP. MM-CBM utilizes dual Concept Bottleneck Layers (CBLs) to align both the image and text embeddings into interpretable features. This allows us to perform new vision tasks like zero-shot classification or image retrieval in an interpretable way. Compared to existing methods, MM-CBM achieves up to 51.26% accuracy improvement on average across four standard benchmarks. Our method maintains high accuracy, staying within ~5% of black-box performance while offering greater interpretability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。