通过权重缩放提升神经网络抗位翻转能力，无需额外计算开销。
Harden Deep Neural Networks Against Fault Injections Through Weight Scaling
- 用常数缩放权重再存储，读取时还原，降低位翻转影响。
- 8位定点模型在0.0001错误率下准确率提升54.418点。
- 适合对可靠性要求高的嵌入式深度学习部署场景。
深度神经网络（DNN）在硬件设备上实现了智能应用，但这些设备易受老化、温度变化和写入错误引起的非预期故障影响。此类故障会导致DNN权重发生位翻转，显著降低性能。因此，在关键应用中保护DNN免受故障影响至关重要。此前工作采用基于纠错码的方法，但通常带来高内存与计算开销。本文提出一种简单有效的权重加固方法：在将权重存入易出错介质前乘以常数，读取时再除以相同常数恢复原尺度。该方法基于观察——位翻转误差具有类似加性噪声的特性，通过缩放可减小绝对误差。我们在四个ImageNet 2012预训练模型上进行实验，涵盖32位浮点、16位浮点和8位定点三种数据类型。结果表明，仅通过权重乘常数操作，8位定点ResNet50在0.0001位错误率下的Top-1准确率提升54.418点。
原文摘要 · Abstract (English)
Deep neural networks (DNNs) have enabled smart applications on hardware devices. However, these hardware devices are vulnerable to unintended faults caused by aging, temperature variance, and write errors. These faults can cause bit-flips in DNN weights and significantly degrade the performance of DNNs. Thus, protection against these faults is crucial for the deployment of DNNs in critical applications. Previous works have proposed error correction codes based methods, however these methods often require high overheads in both memory and computation. In this paper, we propose a simple yet effective method to harden DNN weights by multiplying weights by constants before storing them to fault-prone medium. When used, these weights are divided back by the same constants to restore the original scale. Our method is based on the observation that errors from bit-flips have properties similar to additive noise, therefore by dividing by constants can reduce the absolute error from bit-flips. To demonstrate our method, we conduct experiments across four ImageNet 2012 pre-trained models along with three different data types: 32-bit floating point, 16-bit floating point, and 8-bit fixed point. This method demonstrates that by only multiplying weights with constants, Top-1 Accuracy of 8-bit fixed point ResNet50 is improved by 54.418 at bit-error rate of 0.0001.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。