提出端到端容错注意力机制，提升Transformer在高算力环境下的可靠性与效率。
FT-Transformer: Resilient and Reliable Transformer with End-to-End Fault Tolerant Attention
- 将注意力模块视为单一内核，实现模块级统一容错保护。
- 相比传统方法速度提升最高7.56倍，平均开销仅13.9%。
- 适用于大规模Transformer模型，尤其适合高可靠性要求的推理场景。
Transformer模型依赖高性能计算（HPC）资源进行推理，而大规模系统中软错误不可避免，模型可靠性尤为关键。现有故障容错框架针对操作级别设计，缺乏架构优化，导致显著的计算与内存开销，降低保护效率并限制向更大模型的扩展。本文通过将注意力模块内的操作视为单一内核，实现端到端故障容错的模块级保护，提供跨多步计算的统一防护，并全面覆盖非线性计算中的潜在错误。对于线性模块，设计基于步长算法的容错（ABFT），避免线程间通信。实验表明，该端到端容错方法相比传统方案最高可提速7.56倍，平均故障容错开销仅为13.9%。
原文摘要 · Abstract (English)
Transformer models rely on High-Performance Computing (HPC) resources for inference, where soft errors are inevitable in large-scale systems, making the reliability of the model particularly critical. Existing fault tolerance frameworks for Transformers are designed at the operation level without architectural optimization, leading to significant computational and memory overhead, which in turn reduces protection efficiency and limits scalability to larger models. In this paper, we implement module-level protection for Transformers by treating the operations within the attention module as a single kernel and applying end-to-end fault tolerance. This method provides unified protection across multi-step computations, while achieving comprehensive coverage of potential errors in the nonlinear computations. For linear modules, we design a strided algorithm-based fault tolerance (ABFT) that avoids inter-thread communication. Experimental results show that our end-to-end fault tolerance achieves up to 7.56x speedup over traditional methods with an average fault tolerance overhead of 13.9%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。