提出解析异源注意力机制的通用方法，揭示多源信息融合的模型决策逻辑。
Generic Interpretation Approach for Transformer Models Incorporating Heterogenous Attention Structures
- 针对多源输入设计可解释性方法，捕捉异源注意力的交互机制。
- 在代表性模型上验证了语义与逻辑层面的解释有效性。
- 适合关注多模态模型可解释性的研究人员和应用开发者。
Transformer 极大地推动了人工智能及智能体的发展。我们根据输入信息来源，将 Transformer 的注意力结构分为同源与异源两类。异源注意力结构（如共注意力）能处理来自不同源的信息，是实现复杂功能和融合多模态数据的基础。无论是研究需求还是政策要求，对具备异源注意力结构的 Transformer 模型进行解释都至关重要。多源信息融合带来了新的挑战。本工作主要包括方法与实验两部分：方法上，提出一种针对异源注意力结构的解释方法；实验上，基于所提出的分析范式，对代表性模型进行机制解读，开展语义与逻辑层面的解释。
原文摘要 · Abstract (English)
Transformer has significantly propelled the development of artificial intelligence, and certainly the development of agents as well. We categorize attention structures of Transformer into two types based on the source of the input information: homogenous and heterogenous attention structures. Heterogenous attention structures, with co-attention as a typical example, process information from different sources. Heterogenous attention structure is the foundation for Transformer models to achieve more complex functions and integrate more modal information. Whether for research purposes or policy requirements, the interpretation of Transformer models with heterogenous attention structures is an important task. The fusion of information from different sources brings new challenges. Our work mainly includes two parts: method and experimentation. In terms of method, we propose an interpretation method for Transformer models with heterogenous attention structures. In terms of experimentation, based on our experimental analysis paradigm, we interpret the operating mechanisms of representative models, conduct semantic interpretation and logical interpretation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。