通过优化整数推理中的缩放操作，显著降低硬件开销且不损失模型精度。
Rescaling-Aware Training for Efficient Deployment of Deep Learning Models on Full-Integer Hardware
- 在训练中感知缩放操作，针对缩放因子采用更强量化策略。
- 缩放器宽度减少8倍后，仅需微调即可保持全精度。
- 适合资源受限嵌入式设备的高效低功耗部署场景。
整数推理显著降低了嵌入式系统中的计算复杂度。量化感知训练（QAT）虽能缓解后训练量化带来的精度下降，但仍忽略推理过程中整数缩放这一高成本操作的影响。本文表明，通过在后训练阶段对缩放乘数施加强量化，可大幅降低缩放开销，且无需牺牲模型性能。此外，我们提出一种名为Rescale-Aware Training的微调方法，用于超低比特位宽的缩放乘数。实验显示，即使缩放器宽度减少8倍，通过极小量的增量重训练，仍可保持完整精度。该方法为资源受限嵌入式系统提供了更节能、更高效的AI推理方案。
原文摘要 · Abstract (English)
Integer AI inference significantly reduces computational complexity in embedded systems. Quantization-aware training (QAT) helps mitigate accuracy degradation associated with post-training quantization but still overlooks the impact of integer rescaling during inference, which is a hardware costly operation in integer-only AI inference. This work shows that rescaling cost can be dramatically reduced post-training, by applying a stronger quantization to the rescale multiplicands at no model-quality loss. Furthermore, we introduce Rescale-Aware Training, a fine tuning method for ultra-low bit-width rescaling multiplicands. Experiments show that even with 8x reduced rescaler widths, the full accuracy is preserved through minimal incremental retraining. This enables more energy-efficient and cost-efficient AI inference for resource-constrained embedded systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。