面向边缘设备的Transformer加速架构，兼顾微调与推理效率。
Atleus: Accelerating Transformers on the Edge Enabled by 3D Heterogeneous Manycore Architectures
- 3D异构架构融合非易失内存与流水线阵列，专为Transformer计算优化。
- 实测性能提升最高达56倍，能效比提升64.5倍。
- 适合需要模型压缩与边缘部署的AI应用开发者。
Transformer架构已成为自然语言处理和计算机视觉等众多机器学习应用的标准模型。然而，其高昂的计算与内存需求使得在边缘设备上部署面临挑战。此外，对预训练Transformer（如基础模型）进行微调是提升特定任务性能的常见做法，但现有加速器未能考虑微调带来的复杂性。本文提出一种三维（3D）异构架构Atleus，通过集成3D平台中的非易失性存储与流水线阵列，专门优化Transformer的计算核心以支持微调与推理双重场景。同时，设计了高效片上网络（NoC）以实现高性能与低功耗，并采用有效的量化方案支持模型压缩。实验结果表明，Atleus在性能上相比现有最先进方案最高提升56倍，在能效方面最高提升64.5倍。
原文摘要 · Abstract (English)
Transformer architectures have become the standard neural network model for various machine learning applications including natural language processing and computer vision. However, the compute and memory requirements introduced by transformer models make them challenging to adopt for edge applications. Furthermore, fine-tuning pre-trained transformers (e.g., foundation models) is a common task to enhance the model's predictive performance on specific tasks/applications. Existing transformer accelerators are oblivious to complexities introduced by fine-tuning. In this paper, we propose the design of a three-dimensional (3D) heterogeneous architecture referred to as Atleus that incorporates heterogeneous computing resources specifically optimized to accelerate transformer models for the dual purposes of fine-tuning and inference. Specifically, Atleus utilizes non-volatile memory and systolic array for accelerating transformer computational kernels using an integrated 3D platform. Moreover, we design a suitable NoC to achieve high performance and energy efficiency. Finally, Atleus adopts an effective quantization scheme to support model compression. Experimental results demonstrate that Atleus outperforms existing state-of-the-art by up to 56x and 64.5x in terms of performance and energy efficiency respectively
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。