解决概念瓶颈模型的四大缺陷，提升可解释性与准确性。
Rethinking Concept Bottleneck Models: From Pitfalls to Solutions
- 用熵值度量概念集对数据集的适配度，提前评估概念有效性。
- 引入非线性层修复线性问题，确保模型准确率反映概念相关性。
- 通过教师引导的蒸馏损失缩小与黑箱模型的性能差距，适合可解释性研究者。
概念瓶颈模型（CBMs）将预测基于人类可理解的概念，但存在根本性局限：缺乏预评估概念相关性的度量、因“线性问题”导致模型绕过概念瓶颈、准确率低于黑箱模型，以及缺乏对不同视觉主干网络和视觉语言模型影响的系统研究。本文提出CBM-Suite方法框架，系统应对上述挑战：首先，提出基于熵的度量来量化概念集对特定数据集的内在适用性；其次，通过在概念激活与分类器间插入非线性层，解决线性问题，确保模型准确率真实反映概念相关性；第三，利用线性教师探针引导的蒸馏损失缩小准确率差距；最后，全面分析不同视觉编码器、视觉语言模型与概念集之间的交互如何影响模型准确率与可解释性。大量实验表明，CBM-Suite能生成更准确的模型，并为提升概念驱动的可解释性提供洞察。
原文摘要 · Abstract (English)
Concept Bottleneck Models (CBMs) ground predictions in human-understandable concepts but face fundamental limitations: the absence of a metric to pre-evaluate concept relevance, the "linearity problem" causing recent CBMs to bypass the concept bottleneck entirely, an accuracy gap compared to opaque models, and finally the lack of systematic study on the impact of different visual backbones and VLMs. We introduce CBM-Suite, a methodological framework to systematically addresses these challenges. First, we propose an entropy-based metric to quantify the intrinsic suitability of a concept set for a given dataset. Second, we resolve the linearity problem by inserting a non-linear layer between concept activations and the classifier, which ensures that model accuracy faithfully reflects concept relevance. Third, we narrow the accuracy gap by leveraging a distillation loss guided by a linear teacher probe. Finally, we provide comprehensive analyses on how different vision encoders, vision-language models, and concept sets interact to influence accuracy and interpretability in CBMs. Extensive evaluations show that CBM-Suite yields more accurate models and provides insights for improving concept-based interpretability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。