通过优化参数空间平坦度，提升神经网络对内存位翻转的鲁棒性。
Hessian-aware Training for Enhancing DNNs Resilience to Parameter Corruptions
- 基于海森矩阵设计训练方法，使模型损失曲面更平坦。
- 在多个模型上实现90%~100%精度下降所需翻转位数减少20%~50%。
- 可与硬件级防御协同，适合部署于易受位错影响的边缘设备。
深度神经网络对参数错误极为敏感：内存中单个比特位翻转即可导致精度下降超过10%，最坏情况下达99%。这在存在软件攻击或自然比特翻转的计算平台上构成严峻挑战。现有工作多依赖硬件或系统级防护，但因需平台级改造而难以推广。本文提出一种新方法——海森感知训练，通过提升模型损失曲面平坦度来增强对参数位翻转的鲁棒性。尽管已有基于海森的泛化优化方法，但无法提升抗错能力。相比之下，本方法显著降低敏感比特数，使90%~100%精度下降所需的单比特翻转数量减少20%~50%。同时，该方法与现有硬件防御具协同增益，适用于资源受限且易受比特错误影响的边缘部署场景。
原文摘要 · Abstract (English)
Deep neural networks are not resilient to parameter corruptions: even a single-bitwise error in their parameters in memory can cause an accuracy drop of over 10%, and in the worst cases, up to 99%. This susceptibility poses great challenges in deploying models on computing platforms, where adversaries can induce bit-flips through software or bitwise corruptions may occur naturally. Most prior work addresses this issue with hardware or system-level approaches, such as integrating additional hardware components to verify a model's integrity at inference. However, these methods have not been widely deployed as they require infrastructure or platform-wide modifications. In this paper, we propose a new approach to addressing this issue: training models to be more resilient to bitwise corruptions to their parameters. Our approach, Hessian-aware training, promotes models with $flatter$ loss surfaces. We show that, while there have been training methods, designed to improve generalization through Hessian-based approaches, they do not enhance resilience to parameter corruptions. In contrast, models trained with our method demonstrate increased resilience to parameter corruptions, particularly with a 20$-$50% reduction in the number of bits whose individual flipping leads to a 90$-$100% accuracy drop. Moreover, we show the synergy between ours and existing hardware and system-level defenses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。