arXiv:2411.14733cs.LGcs.SY2024-11

无需重训练,用低精度模拟电路加速Transformer,又快又省电。

FLARE: FP-Less PTQ and Low-ENOB ADC Based AMS-PiM for Error-Resilient, Fast, and Efficient Transformer Acceleration

  • 摒弃传统量化流程,直接在模拟内存中计算,省去浮点单元和高精度转换器
  • 采用低有效位数ADC与稀疏矩阵乘法,能耗降低80%以上,延迟减少65%
  • 适合部署大模型推理,尤其对功耗敏感的边缘设备

基于编码器的Transformer凭借自注意力机制实现了上下文感知表示,但其计算与内存需求呈二次增长,带来显著瓶颈。模拟混合信号存内计算(AMS-PiM)通过片上高效处理缓解此问题。传统AMS-PiM依赖量化感知训练(QAT),需大量重训练,难以适应Transformer;后训练量化(PTQ)虽降低训练开销,却依赖去量化-量化(DQ-Q)过程、浮点单元(FPU)及高有效位数(ENOB)ADC,而高ENOB ADC面积与能耗随2^ENOB指数增长,降低传感裕量,加剧工艺电压温度(PVT)变化影响。为此,本文提出RAP架构:消除DQ-Q流程,实现无FPU与除法的非线性处理,结合低ENOB ADC的稀疏矩阵向量乘法。实验表明,RAP在保持数值稳定的同时,提升容错性、能效与速度,相比先进GPU与传统PiM架构,在能效、延迟与精度上全面领先,为Transformer高效部署提供可扩展方案。

原文摘要 · Abstract (English)

Encoder-based transformers, powered by self-attention layers, have revolutionized machine learning with their context-aware representations. However, their quadratic growth in computational and memory demands presents significant bottlenecks. Analog-Mixed-Signal Process-in-Memory (AMS-PiM) architectures address these challenges by enabling efficient on-chip processing. Traditionally, AMS-PiM relies on Quantization-Aware Training (QAT), which is hardware-efficient but requires extensive retraining to adapt models to AMS-PiMs, making it increasingly impractical for transformer models. Post-Training Quantization (PTQ) mitigates this training overhead but introduces significant hardware inefficiencies. PTQ relies on dequantization-quantization (DQ-Q) processes, floating-point units (FPUs), and high-ENOB (Effective Number of Bits) analog-to-digital converters (ADCs). Particularly, High-ENOB ADCs scale exponentially in area and energy ($2^{ENOB}$), reduce sensing margins, and increase susceptibility to process, voltage, and temperature (PVT) variations, further compounding PTQ's challenges in AMS-PiM systems. To overcome these limitations, we propose RAP, an AMS-PiM architecture that eliminates DQ-Q processes, introduces FPU- and division-free nonlinear processing, and employs a low-ENOB-ADC-based sparse Matrix Vector multiplication technique. Using the proposed techniques, RAP improves error resiliency, area/energy efficiency, and computational speed while preserving numerical stability. Experimental results demonstrate that RAP outperforms state-of-the-art GPUs and conventional PiM architectures in energy efficiency, latency, and accuracy, making it a scalable solution for the efficient deployment of transformers.

存内计算Transformer加速低功耗设计模拟电路

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。