无需批量归一化即可实现全整数量化，提升边缘设备部署可行性
Batch Normalization-Free Fully Integer Quantized Neural Networks via Progressive Tandem Learning
- 分层渐进式蒸馏训练，逐步替代批量归一化
- 在ImageNet上用AlexNet实现高精度全整数推理
- 可直接集成到现有量化流程，适合资源受限设备
量化神经网络（QNN）通过低比特运算缩小模型并降低推理能耗，但多数仍依赖运行统计的批量归一化（BN）层，阻碍真正的全整数部署。以往方法通过参数折叠或特定初始化移除BN，但难以恢复其稳定性和精度，且常需定制约束。本文提出一种无需BN的全整数量化神经网络，采用分层渐进蒸馏方案，从预训练的带BN教师模型出发，利用逐层目标和渐进补偿训练学生模型，实现仅用整数运算进行推理且不包含任何BN操作。在ImageNet数据集上，使用AlexNet在极端量化条件下仍保持竞争力的Top-1精度。该流程可无缝集成至标准量化工作流，支持资源受限场景（如边缘与嵌入式设备）的端到端全整数推理。
原文摘要 · Abstract (English)
Quantised neural networks (QNNs) shrink models and reduce inference energy through low-bit arithmetic, yet most still depend on a running statistics batch normalisation (BN) layer, preventing true integer-only deployment. Prior attempts remove BN by parameter folding or tailored initialisation; while helpful, they rarely recover BN's stability and accuracy and often impose bespoke constraints. We present a BN-free, fully integer QNN trained via a progressive, layer-wise distillation scheme that slots into existing low-bit pipelines. Starting from a pretrained BN-enabled teacher, we use layer-wise targets and progressive compensation to train a student that performs inference exclusively with integer arithmetic and contains no BN operations. On ImageNet with AlexNet, the BN-free model attains competitive Top-1 accuracy under aggressive quantisation. The procedure integrates directly with standard quantisation workflows, enabling end-to-end integer-only inference for resource-constrained settings such as edge and embedded devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。