用模拟数字混合芯片加速Transformer注意力，降低75%计算量且不损失精度。
An Analog and Digital Hybrid Attention Accelerator for Transformers with Charge-based In-memory Computing
- 模拟存内计算动态剪枝75%低分令牌，大幅减少计算
- 仅对25%关键令牌用数字单元精确计算，保持模型准确率
- 在65nm工艺下实现14.8 TOPS/W能效，适合边缘端部署
注意力机制是Transformer的核心计算单元，需对整个输入序列进行成对相关性计算。自注意力计算复杂度高且频繁访问内存，随着序列长度增加带来巨大系统负担。本文提出一种65nm CMOS工艺下的模拟与数字混合处理器，用于加速Transformer的注意力机制。设计了一种模拟存内计算（CIM）核心，在运行时平均剪枝约75%低分令牌，实现超低功耗和延迟。同时，数字处理器仅对约25%未被剪枝的令牌进行精确计算，避免精度下降。实测结果表明，模拟核心峰值能效达14.8 TOPS/W，系统级（SoC）为1.65 TOPS/W；模拟核心面积效率达976.6 GOPS/mm²，系统级为79.4 GOPS/mm²。
原文摘要 · Abstract (English)
The attention mechanism is a key computing kernel of Transformers, calculating pairwise correlations across the entire input sequence. The computing complexity and frequent memory access in computing self-attention put a huge burden on the system especially when the sequence length increases. This paper presents an analog and digital hybrid processor to accelerate the attention mechanism for transformers in 65nm CMOS technology. We propose an analog computing-in-memory (CIM) core, which prunes ~75% of low-score tokens on average during runtime at ultra-low power and delay. Additionally, a digital processor performs precise computations only for ~25% unpruned tokens selected by the analog CIM core, preventing accuracy degradation. Measured results show peak energy efficiency of 14.8 and 1.65 TOPS/W, and peak area efficiency of 976.6 and 79.4 GOPS/mm$^\mathrm{2}$ in the analog core and the system-on-chip (SoC), respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。