根据计算强度动态调整每层精度，提升模型推理速度
Arithmetic-Intensity-Aware Quantization
- 按层智能分配比特数，提升计算密度以突破内存瓶颈
- 在CIFAR-10上计算强度提升50%，准确率仅降1个百分点
- 适合移动端等内存受限场景的高效推理部署
随着现代神经网络日益成为内存瓶颈，推理吞吐量受限于DRAM带宽而非计算能力。本文提出算术强度感知量化（AIQ），一种混合精度量化框架，通过为每层选择最优比特数，在最小化精度损失的同时最大化算术强度（AI）。AIQ是一种后训练量化方法，采用搜索算法在每层量化方案中寻找加权损失最小的配置，权衡算术强度与精度。在ResNet-20/CIFAR-10上，相比FP32基线，AIQ使算术强度提升约50%，测试准确率仅下降约1个百分点，优于全局统一量化方案。在内存受限的MobileNetV2架构上，AIQ配置实现1.66倍于FP32基线的吞吐量，同时保持准确率在1个百分点内。此外发现，AIQ会自然对大层进行更激进的量化。
原文摘要 · Abstract (English)
As modern neural networks become increasingly memory-bound, inference throughput is limited by DRAM bandwidth rather than compute. We present Arithmetic-Intensity-Aware Quantization (AIQ), a mixed precision quantization framework that chooses per-layer bit-widths to maximize arithmetic intensity (AI) while minimizing accuracy loss. AIQ is a post-training quantization method that uses search algorithms over per-layer quantization schemes to minimize a weighted loss over AI and accuracy. On ResNet-20/CIFAR-10, AIQ increases AI by ~50% over an FP32 baseline while keeping test accuracy within ~1 percentage point, and outperforming global uniform quantization schemes. On a memory-bound MobileNetV2 architecture, AIQ configurations give a 1.66x higher throughput than the FP32 baseline while keeping test accuracy within 1 percentage point. We also find that AIQ naturally quantizes larger layers more aggressively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。