让神经网络支持运行时灵活调整权重位宽，提升边缘设备部署效率。
TruncQuant: Truncation-Ready Quantization for DNNs with Flexible Weight Bit Precision
- 训练时对权重进行截断友好设计，支持运行时动态调整位宽。
- 在2~8位精度下保持模型准确率稳定，误差小于1%。
- 兼容现有量化框架，适合需要跨硬件部署的模型优化场景。
将深度神经网络部署到边缘设备面临模型复杂度高、存储与推理延迟大的挑战。近期研究通过多样化量化设置寻找计算效率与精度的平衡点。截断是一种实现低比特映射的有效方法，可使单一模型适配多种硬件平台，且开销极小。然而，如何设计训练方案以应对截断带来的误差，仍是难题，因现有量化感知训练方法未针对截断过程优化。本文提出 TruncQuant，一种新型截断就绪训练方案，支持运行时通过位移操作实现灵活权重位宽。该方法通过与截断输出对齐，展现出在不同位宽下的强鲁棒性，并可在现有量化感知框架中轻松实现。代码已开源：https://github.com/a2jinhee/TruncQuant。
原文摘要 · Abstract (English)
The deployment of deep neural networks on edge devices is a challenging task due to the increasing complexity of state-of-the-art models, requiring efforts to reduce model size and inference latency. Recent studies explore models operating at diverse quantization settings to find the optimal point that balances computational efficiency and accuracy. Truncation, an effective approach for achieving lower bit precision mapping, enables a single model to adapt to various hardware platforms with little to no cost. However, formulating a training scheme for deep neural networks to withstand the associated errors introduced by truncation remains a challenge, as the current quantization-aware training schemes are not designed for the truncation process. We propose TruncQuant, a novel truncation-ready training scheme allowing flexible bit precision through bit-shifting in runtime. We achieve this by aligning TruncQuant with the output of the truncation process, demonstrating strong robustness across bit-width settings, and offering an easily implementable training scheme within existing quantization-aware frameworks. Our code is released at https://github.com/a2jinhee/TruncQuant.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。