针对Transformer加速器的线性投影数据重用，提出按块自适应选择存储策略。
An Efficient Data Reuse with Tile-Based Adaptive Stationary for Transformer Accelerators
- 按输入序列长度动态选择输入或权重在块上的驻留策略
- 相比传统方案降低97%以上的外部内存访问开销
- 兼容多种注意力优化技术与硬件加速器
基于Transformer的模型已成为计算机视觉和自然语言处理等领域的主流架构。然而，随着模型规模增大，权重与激活值的外部内存访问（EMA）因能耗远高于内部计算，成为关键瓶颈。尽管多数研究聚焦于优化自注意力机制，但对线性投影中数据传输的优化关注较少，而此处的EMA开销同样显著。本文提出基于块的自适应驻留（Tile-based Adaptive Stationary, TAS）方案，根据输入序列长度在块粒度上动态选择输入或权重驻留策略。实验表明，TAS相比传统驻留方案可将EMA降低超过97%，且兼容多种注意力优化技术与硬件加速器。
原文摘要 · Abstract (English)
Transformer-based models have become the \textit{de facto} backbone across many fields, such as computer vision and natural language processing. However, as these models scale in size, external memory access (EMA) for weight and activations becomes a critical bottleneck due to its significantly higher energy consumption compared to internal computations. While most prior work has focused on optimizing the self-attention mechanism, little attention has been given to optimizing data transfer during linear projections, where EMA costs are equally important. In this paper, we propose the Tile-based Adaptive Stationary (TAS) scheme that selects the input or weight stationary in a tile granularity, based on the input sequence length. Our experimental results demonstrate that TAS can significantly reduce EMA by more than 97\% compared to traditional stationary schemes, while being compatible with various attention optimization techniques and hardware accelerators.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。