发现神经网络内存错误敏感度差异,实现按需保护以大幅降低纠错开销。
From Bit-Position Sensitivity to Unequal Error Protection for DNN Inference Memory

- 按位敏感度分析揭示低阶比特可忽略,高阶与指数位至关重要。
- 设计不均等纠错方案,使纠错开销降低27.8%,节能约17%。
- 适用于大模型推理加速器,尤其适合文本生成与视觉编码器场景。
我们对16个工作负载(包括基于Transformer的模型和无注意力CNN)在三种浮点格式下的每比特位故障敏感度进行了全面评估。核心发现是存在显著的比特敏感度跃迁:在数据类型特定阈值Xsafe以内翻转最低有效分数位,任务指标下降小于1%。敏感度随高位分数位上升,在指数-尾数边界处单比特翻转引发灾难性崩溃。因此,均匀的SECDED保护(12.5%存储开销)过于保守。我们推导出各数据类型的Xsafe下限(FP16: 6, BF16: 4, FP32: 15),并设计面向工作负载的分层保护机制,使可免保护区域扩大,无需重训练即可实现37.5%-62.5%的纠错码节省。文本条件扩散模型设定保守下限;视觉编码器、NLU模型及鲁棒大模型容忍更宽的免保护区。该方案驱动一种带每缓存行数据类型标签的不均等错误保护(UEP)编码器,配合双分区SRAM架构。870多次故障注入验证表明,选择性保护在连续2-3比特故障下依然有效。相比统一SECDED,该编码器减少27.8%纠错电路面积;非关键分区双电压运行使BF16读取功耗降低约17%,双分区宏单元面积增加约4%。
原文摘要 · Abstract (English)
We characterize per-bit-position fault sensitivity in ML inference across 16 workloads -- spanning transformer-based models and attention-free CNNs -- and across three floating-point formats. Our central empirical finding is a sharp bit-sensitivity transition: flipping any of the least-significant fraction bits up to a data-type-specific threshold, Xsafe, degrades task metrics by less than 1% under deterministic single-bit stress tests. Sensitivity rises through the upper fraction bits and spikes at the exponent-mantissa boundary, where a single-bit flip causes catastrophic collapse. Because low-order bits are largely inconsequential while high-order and exponent bits are critical, uniform SECDED protection -- which guards every bit equally at 12.5% storage overhead -- is unnecessarily conservative. We derive per-data-type Xsafe floors (FP16: 6, BF16: 4, FP32: 15) and workload-aware tiers that widen the unprotected region for resilient model classes, raising ECC savings to 37.5-62.5% without retraining. Text-conditioned diffusion models dictate the conservative floor; vision encoders, NLU models, and resilient LLMs tolerate wider bypass regions. These floors and tiers drive an Unequal Error Protection (UEP) codec with per-cacheline data-type tags and a dual-partition SRAM architecture for ML accelerators. Validation across 870+ fault-injection runs confirms selective protection holds under contiguous 2- and 3-bit upsets. The codec reduces ECC area by 27.8% relative to uniform SECDED; dual-voltage operation of the non-critical partition lowers gross BF16 read energy by about 17%, with a roughly 4% dual-partition macro-area overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。