arXiv:2506.03370cs.LGcs.CL2025-06
比较不同硬注意力变换器对形式语言的识别能力
Comparison of different Unique hard attention transformer models by the formal languages they can recognize
- 区分遮蔽与非遮蔽、有限与无限输入等模型变体
- 给出一阶逻辑下界和电路复杂度上界
- 适合形式语言与注意力机制交叉研究者阅读
本文综述了各类唯一硬注意力变换器编码器（UHATs）在识别形式语言方面的能力。区分了遮蔽与非遮蔽、有限与无限输入，以及通用与双线性注意力评分函数。回顾了这些模型之间的关系，并给出了基于一阶逻辑的下界与基于电路复杂度的上界。
原文摘要 · Abstract (English)
This note is a survey of various results on the capabilities of unique hard attention transformers encoders (UHATs) to recognize formal languages. We distinguish between masked vs. non-masked, finite vs. infinite image and general vs. bilinear attention score functions. We recall some relations between these models, as well as a lower bound in terms of first-order logic and an upper bound in terms of circuit complexity.
形式语言注意力机制变换器
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。