提出细粒度概念注意力机制，提升图像与概念对齐精度。
Concept-wise Attention for Fine-grained Concept Bottleneck Models
- 用可学习的概念视觉查询实现细粒度特征提取
- 引入概念对比优化，解决概念间的互斥性问题
- 显著提升模型解释性，适合需要可解释性的场景
近年来，利用大型预训练视觉语言模型（如CLIP）学习的图像-文本对齐能力，显著提升了概念瓶颈模型（CBM）的性能。然而，现有方法在概念建模中仍存在两大局限：一是存在预训练偏差，表现为粒度不匹配或依赖结构先验；二是使用二元交叉熵（BCE）损失微调时，各概念独立处理，忽略了概念间的互斥性，导致对齐效果不佳。为此，本文提出概念注意力细粒度概念瓶颈模型（CoAt-CBM），通过可学习的概念级视觉查询自适应获取细粒度概念视觉嵌入，并生成概念得分向量；进一步设计概念对比优化策略，引导模型合理分配概念得分相对重要性，使预测更忠实于图像内容并提升对齐效果。大量实验表明，CoAt-CBM持续优于当前最优方法。代码将在录用后公开。
原文摘要 · Abstract (English)
Recently impressive performance has been achieved in Concept Bottleneck Models (CBM) by utilizing the image-text alignment learned by a large pre-trained vision-language model (i.e. CLIP). However, there exist two key limitations in concept modeling. Existing methods often suffer from pre-training biases, manifested as granularity misalignment or reliance on structural priors. Moreover, fine-tuning with Binary Cross-Entropy (BCE) loss treats each concept independently, which ignores mutual exclusivity among concepts, leading to suboptimal alignment. To address these limitations, we propose Concept-wise Attention for Fine-grained Concept Bottleneck Models (CoAt-CBM), a novel framework that achieves adaptive fine-grained image-concept alignment and high interpretability. Specifically, CoAt-CBM employs learnable concept-wise visual queries to adaptively obtain fine-grained concept-wise visual embeddings, which are then used to produce a concept score vector. Then, a novel concept contrastive optimization guides the model to handle the relative importance of the concept scores, enabling concept predictions to faithfully reflect the image content and improved alignment. Extensive experiments demonstrate that CoAt-CBM consistently outperforms state-of-the-art methods. The codes will be available upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。