通过插入去噪模块，显著提升模拟神经网络在噪声下的推理准确率。
Efficient Noise Mitigation for Enhancing Inference Accuracy in DNNs on Mixed-Signal Accelerators
- 在预训练模型中插入可训练的去噪块，缓解模拟器件老化与工艺偏差带来的噪声影响。
- 在ImageNet和CIFAR-10上，噪声导致的准确率下降从31.7%降至1.15%。
- 提出高效架构与插入点优化算法，适合部署于混合信号加速器。
本文提出一种框架，通过缓解模拟计算元件因工艺差异和老化引起的变异性对神经网络精度的影响，增强模型鲁棒性。将这些变异性建模为影响激活精度的噪声，并在预训练模型的选定层之间插入去噪块。实验表明，训练去噪块能显著提升模型对多种噪声水平的鲁棒性。为降低添加去噪块的开销，提出一种探索算法以确定最优插入位置，并设计专用架构以高效执行去噪块，可集成至混合信号加速器。在ImageNet和CIFAR-10数据集上评估结果显示，仅增加2.03%参数量，噪声导致的准确率下降由31.7%降至1.15%。
原文摘要 · Abstract (English)
In this paper, we propose a framework to enhance the robustness of the neural models by mitigating the effects of process-induced and aging-related variations of analog computing components on the accuracy of the analog neural networks. We model these variations as the noise affecting the precision of the activations and introduce a denoising block inserted between selected layers of a pre-trained model. We demonstrate that training the denoising block significantly increases the model's robustness against various noise levels. To minimize the overhead associated with adding these blocks, we present an exploration algorithm to identify optimal insertion points for the denoising blocks. Additionally, we propose a specialized architecture to efficiently execute the denoising blocks, which can be integrated into mixed-signal accelerators. We evaluate the effectiveness of our approach using Deep Neural Network (DNN) models trained on the ImageNet and CIFAR-10 datasets. The results show that on average, by accepting 2.03% parameter count overhead, the accuracy drop due to the variations reduces from 31.7% to 1.15%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。