通过注意力瓶颈实现视觉语言模型的可解释干预与调试
TAB: Transformer Attention Bottlenecks enable User Intervention and Debugging in Vision-Language Models
- 引入单头注意力瓶颈，约束总注意力在[0,1]区间，实现可控信息传递
- 在三组数据集上保持生成质量，同时显著提升变化定位与无变化识别能力
- 首次支持用户编辑注意力以调试模型，输出更符合预期
多头自注意力（MHSA）是语言与视觉领域广泛使用的Transformer架构的核心组件。尽管多头设计使模型能并行处理输入，但也模糊了各图像块对输出的贡献归属。本文提出一种新型单头注意力瓶颈（TAB）层，插入传统MHSA之后，用于提升模型可解释性与可干预性。与标准自注意力不同，TAB将所有图像块的总注意力约束在[0,1]范围内：当总注意力为0时，不传递任何视觉信息，模型将默认生成与图像无关的通用响应。为验证优势，我们在图像差异描述任务中训练含TAB的视觉语言模型。在三个数据集上，其生成质量与基线模型相当，但注意力瓶颈在变化定位和识别无变化方面表现更优。TAB是首个支持用户通过编辑注意力进行调试的架构，常可产生预期输出。
原文摘要 · Abstract (English)
Multi-head self-attention (MHSA) is a key component of Transformers, a widely popular architecture in both language and vision. Multiple heads intuitively enable different parallel processes over the same input. Yet, they also obscure the attribution of each input patch to the output of a model. We propose a novel 1-head Transformer Attention Bottleneck (TAB) layer, inserted after the traditional MHSA architecture, to serve as an attention bottleneck for interpretability and intervention. Unlike standard self-attention, TAB constrains the total attention over all patches to $\in [0, 1]$. That is, when the total attention is 0, no visual information is propagated further into the network, and the vision-language model (VLM) would default to a generic, image-independent response. To demonstrate the advantages of TAB, we train VLMs with TAB to perform image-difference captioning. Over three datasets, our models perform similarly to baseline VLMs in captioning but the bottleneck is superior in localizing changes and in identifying when no changes occur. TAB is the first architecture to enable users to debug by editing attention, which often produces expected outputs by VLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。