用有界激活函数提升嵌入式DNN抗软错误能力,兼顾精度与计算效率。
Designing DNNs for a trade-off between robustness and processing performance in embedded devices
- 采用有界激活函数增强DNN对参数扰动的鲁棒性
- 在KV260上验证了模型在压缩后仍保持高精度和低延迟
- 适合自动驾驶等安全关键场景的嵌入式部署
基于机器学习的嵌入式系统在航空航天、自动驾驶等安全关键应用中需抵御由软错误引起的扰动。现代数字处理器因更小的晶体管尺寸和更低电压,对背景辐射更敏感,软错误风险上升。DNN模型对参数扰动的鲁棒性主要取决于模型结构、数值表示及计算精度。当采用剪枝和量化等压缩技术降低内存占用与计算复杂度时,模型结构与数值表示均被改变,导致软错误鲁棒性随之变化。尽管激活函数(AFs)常被忽视,但其不仅影响模型准确率与可训练性,还决定压缩率与数值鲁棒性。本文研究使用有界激活函数以提升DNN对参数扰动的鲁棒性,同时评估其对部署性能的影响,包括模型准确率、可压缩性与计算开销。特别针对用于自动驾驶场景理解的高光谱图像语义分割任务,分析了编码器-解码器全卷积模型。部署特性通过AMD-Xilinx KV260 SoM实验验证。
原文摘要 · Abstract (English)
Machine learning-based embedded systems employed in safety-critical applications such as aerospace and autonomous driving need to be robust against perturbations produced by soft errors. Soft errors are an increasing concern in modern digital processors since smaller transistor geometries and lower voltages give electronic devices a higher sensitivity to background radiation. The resilience of deep neural network (DNN) models to perturbations in their parameters is determined, to a large extent, by the structure of the model itself, and also by the selected numerical representation and used arithmetic precision. When compression techniques such as model pruning and model quantization are applied to reduce memory footprint and computational complexity for deployment, both model structure and numerical representation are modified and thus, soft error robustness also changes. In this sense, although the choice of activation functions (AFs) in DNN models is frequently ignored, it conditions not only their accuracy and trainability, but also compressibility rates and numerical robustness. This paper investigates the suitability of using bounded AFs to improve model robustness against DNN parameter perturbations, assessing at the same time the impact of this choice on deployment in terms of model accuracy, compressibility, and computational burden. In particular, we analyze encoder-decoder fully convolutional models aimed at performing semantic segmentation tasks on hyperspectral images for scene understanding in autonomous driving. Deployment characterization is performed experimentally on an AMD-Xilinx's KV260 SoM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。