发现Transformer中存在能检测词元重复的特殊注意力头。
The Anxiety of Influence: Bloom Filters in Transformer Attention Heads
- 识别出4个模型中的3个真实成员检测头，具不同精度策略。
- 部分头在180个不重复词元下误报率仅0-4%，超经典布隆过滤器容量。
- 这些头集中于早期层，对任意重复词元均有泛化能力，适合研究模型记忆机制者阅读。
一些Transformer注意力头似乎充当成员测试器，专门回答“该词元是否在上下文中出现过？”我们在四个语言模型（GPT-2 small、medium、large；Pythia-160M）中识别出这类头，并发现它们构成成员检测策略谱系。两个头（GPT-2 small的L0H1和L0H5）表现出高精度成员过滤能力，即使在180个不重复上下文词元下，误报率仍为0-4%，远超经典布隆过滤器的$ d_ ext{head} = 64 $位容量。第三个头（L1H11）展现出经典的布隆过滤器容量曲线，其误报率符合理论公式 $ p \approx (1 - e^{-kn/m})^k $，$ R^2 = 1.0 $，拟合容量 $ m \approx 5 $ 位，约在 $ n \approx 20 $ 个唯一词元时饱和。第四个头（L3H0）最初被误判为布隆过滤器，但在控制混杂因素后被重新归类为通用前缀注意力头，其看似容量曲线实为序列长度伪影。三个真实成员检测头共同构成多分辨率系统，集中于早期层（0-1），与归纳和前词元头在分类上区分明显，且误报率随嵌入距离单调递减——符合距离敏感布隆过滤器特征。这些头具备广泛泛化性：对任意重复词元类型响应，而非仅限重复名称，其泛化能力比仅针对重复词元的头高出43%。消融实验表明，这些头同时参与重复与新词元处理，说明成员检测与更广泛计算功能共存。对L3H0的重新分类反而强化了结论：经受住筛选的存活头更具可信度。
原文摘要 · Abstract (English)
Some transformer attention heads appear to function as membership testers, dedicating themselves to answering the question "has this token appeared before in the context?" We identify these heads across four language models (GPT-2 small, medium, and large; Pythia-160M) and show that they form a spectrum of membership-testing strategies. Two heads (L0H1 and L0H5 in GPT-2 small) function as high-precision membership filters with false positive rates of 0-4\% even at 180 unique context tokens -- well above the $d_\text{head} = 64$ bit capacity of a classical Bloom filter. A third head (L1H11) shows the classic Bloom filter capacity curve: its false positive rate follows the theoretical formula $p \approx (1 - e^{-kn/m})^k$ with $R^2 = 1.0$ and fitted capacity $m \approx 5$ bits, saturating by $n \approx 20$ unique tokens. A fourth head initially identified as a Bloom filter (L3H0) was reclassified as a general prefix-attention head after confound controls revealed its apparent capacity curve was a sequence-length artifact. Together, the three genuine membership-testing heads form a multi-resolution system concentrated in early layers (0-1), taxonomically distinct from induction and previous-token heads, with false positive rates that decay monotonically with embedding distance -- consistent with distance-sensitive Bloom filters. These heads generalize broadly: they respond to any repeated token type, not just repeated names, with 43\% higher generalization than duplicate-token-only heads. Ablation reveals these heads contribute to both repeated and novel token processing, indicating that membership testing coexists with broader computational roles. The reclassification of L3H0 through confound controls strengthens rather than weakens the case: the surviving heads withstand the scrutiny that eliminated a false positive in our own analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。