通过重排运算顺序，实现视觉Transformer的低比特整数推理，降低硬件功耗。
Low-Bit Integerization of Vision Transformers using Operand Reordering for Efficient Hardware
- 延迟解量化至矩阵运算后，直接用量化输入进行整数计算。
- 在硬件上实现低比特推理，线性层与矩阵乘法的每处理单元功耗显著下降。
- 适合追求高效低功耗部署的视觉模型研究者和芯片设计团队。
预训练的视觉变换器在各类视觉任务中表现卓越，但计算和内存开销巨大。尽管模型量化通过降低精度减少了内存占用，但矩阵运算前的解量化仍带来显著计算负担。本文分析计算图，提出基于操作重排的整数化方法：将解量化延迟至矩阵运算之后，从而实现量化输入下的整数矩阵乘法与线性模块。为验证该方法，我们在基于脉动阵列的硬件上合成ViT的自注意力模块。实验表明，低比特推理有效降低了线性层和矩阵乘法的每处理单元（PE）功耗，缩小了量化模型与高效推理之间的差距。
原文摘要 · Abstract (English)
Pre-trained vision transformers have achieved remarkable performance across various visual tasks but suffer from expensive computational and memory costs. While model quantization reduces memory usage by lowering precision, these models still incur significant computational overhead due to the dequantization before matrix operations. In this work, we analyze the computation graph and propose an integerization process based on operation reordering. Specifically, the process delays dequantization until after matrix operations. This enables integerized matrix multiplication and linear module by directly processing the quantized input. To validate our approach, we synthesize the self-attention module of ViT on a systolic array-based hardware. Experimental results show that our low-bit inference reduces per-PE power consumption for linear layer and matrix multiplication, bridging the gap between quantized models and efficient inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。