通过差分计算实现视觉Transformer的低功耗加速
DeVIT: Low-Power Vision Transformer Acceleration Using Delta Computation
- 利用参数值局部性，基于差分计算实现无乘法矩阵乘法
- 在保持精度的同时降低计算复杂度和内存占用
- 适合部署于资源受限设备的视觉Transformer推理
基于Transformer的深度学习模型在自然语言处理和计算机视觉等领域取得了前所未有的性能提升。然而，将其部署在资源受限设备上面临巨大挑战，主要源于其高计算复杂度、大内存占用和带宽需求。为降低处理与内存开销，研究者常采用低比特权重量化。除减少计算量外，量化还引入了值局部性——大量参数被限制在有限数值范围内。为充分利用这一特性，本文提出DeVIT，一种面向视觉Transformer的加速方法，通过差分计算实现无乘法矩阵乘法，显著降低计算能耗。
原文摘要 · Abstract (English)
The emergence of transformer-based deep learning models has brought unprecedented performance across various domains, particularly in natural language processing and computer vision. However, deploying these models, especially on resource-constrained devices, poses significant challenges due to their high computational complexity and large memory size and bandwidth requirements. This complexity has led researchers to use low-bit model weights to reduce memory usage and improve efficiency. In addition to reducing processing and memory demands, quantization introduces another useful property: value locality, where the extremely large number of parameters are restricted to a limited range of values. To fully take advantage of this locality, this paper presents DeVIT, an acceleration method for vision transformers that leverages differential computation to enable multiplier-less matrix multiplication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。