测试大模型组合关系推理能力，发现其存在根本缺陷。
Benchmarking and Understanding Compositional Relational Reasoning of LLMs
- 构建新基准GAR统一评估模型的组合关系推理能力。
- 33B参数模型中发现关键注意力头决定推理正确性。
- 识别出代表真假概念的注意力头，适用于多种任务。
组合关系推理（CRR）是人类智能的特征，但现有基于Transformer的大语言模型（LLMs）是否具备此类能力仍不明确。为此，我们提出一种名为广义关联回忆（GAR）的新合成基准，整合并泛化了机制可解释性（MI）研究中的多个任务，形成统一框架。评估表明，GAR对现有模型具有足够挑战性，暴露出其在CRR上的根本缺陷，同时又适合系统化的MI研究。为理解模型如何解决GAR任务，我们采用归因修补法，发现Vicuna-33B在不同任务中复用一组核心电路和关键注意力头。干预实验显示，这些头的正常运作显著影响性能。尤其发现两类注意力头，其激活分别表征了GAR任务中的“真”与“假”抽象概念，在多种模型与任务中起基础作用。数据集与代码已公开于https://github.com/Caiyun-AI/GAR。
原文摘要 · Abstract (English)
Compositional relational reasoning (CRR) is a hallmark of human intelligence, but we lack a clear understanding of whether and how existing transformer large language models (LLMs) can solve CRR tasks. To enable systematic exploration of the CRR capability of LLMs, we first propose a new synthetic benchmark called Generalized Associative Recall (GAR) by integrating and generalizing the essence of several tasks in mechanistic interpretability (MI) study in a unified framework. Evaluation shows that GAR is challenging enough for existing LLMs, revealing their fundamental deficiency in CRR. Meanwhile, it is easy enough for systematic MI study. Then, to understand how LLMs solve GAR tasks, we use attribution patching to discover the core circuits reused by Vicuna-33B across different tasks and a set of vital attention heads. Intervention experiments show that the correct functioning of these heads significantly impacts task performance. Especially, we identify two classes of heads whose activations represent the abstract notion of true and false in GAR tasks respectively. They play a fundamental role in CRR across various models and tasks. The dataset and code are available at https://github.com/Caiyun-AI/GAR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。