通过融合指数与乘法运算，降低注意力硬件加速器的面积和功耗。
Low-Cost FlashAttention with Fused Exponential and Multiplication Hardware Operators
- 设计新型ExpMul硬件操作，将指数与向量乘法合并计算。
- 在28nm ASIC上实现面积减少28.8%，功耗降低17.6%。
- 适用于需要高效推理的长序列Transformer模型部署场景。
注意力机制，尤其是Transformer架构和大语言模型（LLMs）中的注意力机制，已彻底改变机器学习与人工智能中序列建模的方式。为处理日益增长的序列长度，已有专用加速器被提出，直接在硬件层面执行关键注意力步骤。近年来，基于FlashAttention算法的架构尤为突出，因其优化的计算方式、分块能力及更低的内存流量。本文聚焦于浮点型FlashAttention核的优化，引入新的硬件操作，将指数计算（e^x）与向量乘法（V）融合。所提出的ExpMul硬件操作显著降低了基于FlashAttention的硬件加速器的面积与功耗。在28nm ASIC工艺下，相较于现有独立实现指数与乘法操作的硬件架构，平均面积减少28.8%，功耗降低17.6%。
原文摘要 · Abstract (English)
Attention mechanisms, particularly within Transformer architectures and large language models (LLMs), have revolutionized sequence modeling in machine learning and artificial intelligence applications. To compute attention for increasingly long sequences, specialized accelerators have been proposed to execute key attention steps directly in hardware. Among the various recently proposed architectures, those based on variants of the FlashAttention algorithm, originally designed for GPUs, stand out due to their optimized computation, tiling capabilities, and reduced memory traffic. In this work, we focus on optimizing the kernel of floating-point-based FlashAttention using new hardware operators that fuse the computation of exponentials and vector multiplications, e.g., e^x, V. The proposed ExpMul hardware operators significantly reduce the area and power costs of FlashAttention-based hardware accelerators. When implemented in a 28nm ASIC technology, they achieve improvements of 28.8% in area and 17.6% in power, on average, compared to state-of-the-art hardware architectures with separate exponentials and vector multiplications hardware operators.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。