让视频分类更可解释：用时序概念捕捉动作与物体变化
Concepts in Motion: Temporal Concept Bottleneck Model for Interpretable Video Classification
- 基于视觉语言模型自动提取视频中的动作和物体概念
- 通过时序自注意力建模概念出现的时间模式，提升预测准确性
- 无需人工标注概念，适合需要透明决策的视频分析场景
概念瓶颈模型（CBMs）通过人类可理解的概念实现图像分类的可解释性，但将其扩展至视频仍具挑战，主要源于概念提取困难及时间建模复杂。本文提出MoTIF（Moving Temporal Interpretable Framework），一种基于Transformer的时序概念架构，通过每个概念的时序自注意力机制，建模个体概念的重复出现时机及其对分类的贡献。框架核心为类条件的VLM-based概念发现模块，可从训练视频中自动提取以物体和动作为中心的文本概念，生成无需人工标注的时序表达概念集。在多个视频基准测试中，该方法优于全局概念瓶颈模型，在可解释性设定下保持竞争力，并缩小了与强黑盒视频基线的差距。代码已开源。
原文摘要 · Abstract (English)
Concept Bottleneck Models (CBMs) enable interpretable image classification by structuring predictions around human-understandable concepts, but extending this paradigm to video remains challenging due to the difficulty of extracting concepts and modeling them over time. In this paper, we introduce MoTIF (Moving Temporal Interpretable Framework), a transformer-based concept architecture that operates on sequences of temporally grounded concept activations, by employing per-concept temporal self-attention to model when individual concepts recur and how their temporal patterns contribute to predictions. Central to the framework is a class-conditioned VLM-based concept discovery module that extracts object- and action-centric textual concepts from training videos, yielding temporally expressive concept sets without manual concept annotation. Across multiple video benchmarks, this combination improves over global concept bottlenecks and remains competitive within the interpretable concept-bottleneck setting, while narrowing the gap to strong black-box video baselines that we report as contextual references. Code available at github.com/patrick-knab/MoTIF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。