为注意力层设计高效容错机制，用单次校验覆盖整个计算流程。
Custom Algorithm-based Fault Tolerance for Attention Layers in Transformers
- 通过在线计算三矩阵乘积及softmax的统一校验和，实现全链路容错。
- 仅需5.3%硬件面积开销和1.9%能量损耗，检测精度高。
- 适合部署于大模型加速器，尤其关注能效与可靠性的硬件设计者。
Transformer与大语言模型（LLMs）凭借注意力机制驱动了众多AI应用，推动了专用硬件加速器的发展。加速器面临的一个主要挑战是如何高效检测由随机硬件故障引发的错误。传统基于算法的容错（ABFT）技术仅验证单个矩阵乘法，难以应对完整的注意力机制，尤其是中间的softmax归一化环节。本文提出Flash-ABFT，一种新方法，通过一次校验计算整个注意力层中查询、键、值矩阵三重乘积及softmax操作的在线校验和，显著降低冗余检查带来的开销，同时保持高故障检测精度。实验表明，Flash-ABFT仅引入5.3%的硬件面积开销和低于1.9%的能量开销，是一种成本效益高且可靠的注意力加速器错误检测方案。
原文摘要 · Abstract (English)
Transformers and large language models (LLMs), powered by the attention mechanism, have transformed numerous AI applications, driving the need for specialized hardware accelerators. A major challenge in these accelerators is efficiently detecting errors caused by random hardware faults. Traditional algorithm-based fault tolerance (ABFT) techniques verify individual matrix multiplications but fall short in handling the full attention mechanism, particularly due to intermediate softmax normalization. This work proposes Flash-ABFT, a novel method that computes an online checksum across the entire three-matrix product of query, key and value matrices, of an attention layer, including the softmax operation, with a single check. This approach significantly reduces overhead by eliminating redundant checks while maintaining high fault-detection accuracy. Experimental results demonstrate that Flash-ABFT incurs only 5.3% hardware area overhead and less than 1.9% energy overhead, making it a cost-effective and robust solution for error detection in attention accelerators.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。