通过排序和贪心求和，显著提升8位浮点运算的精度和能效。
MGS: Markov Greedy Sums for Accurate Low-Bitwidth Floating-Point Accumulation
- 按指数排序后贪心累加，避免低位数溢出
- 8位浮点下分类准确率接近32位基准
- 适合低功耗神经网络推理场景
我们提出一种新方法 MGS（Markov Greedy Sums），用于改进神经网络计算中低比特浮点点积的精度。传统32位浮点累加时，不同指数值相加会导致小数值的尾数右移对齐，造成精度损失（称为‘淹没’）。在8位浮点等低比特场景下，尾数位数少，此问题更严重。MGS通过按指数排序并直接累加尾数，避免了大部分淹没误差，且不溢出低比特累加器。我们设计、分析并实现了该算法，在多个神经网络上优化了8位浮点推理的数值精度。相比传统顺序累加，本方法显著降低误差，使图像分类任务的准确率与高精度浮点基线相当。其dMAC硬件单元相较传统MAC单元可降低最高达34.1%的功耗。
原文摘要 · Abstract (English)
We offer a novel approach, MGS (Markov Greedy Sums), to improve the accuracy of low-bitwidth floating-point dot products in neural network computations. In conventional 32-bit floating-point summation, adding values with different exponents may lead to loss of precision in the mantissa of the smaller term, which is right-shifted to align with the larger term's exponent. Such shifting (a.k.a. 'swamping') is a significant source of numerical errors in accumulation when implementing low-bitwidth dot products (e.g., 8-bit floating point) as the mantissa has a small number of bits. We avoid most swamping errors by arranging the terms in dot product summation based on their exponents and summing the mantissas without overflowing the low-bitwidth accumulator. We design, analyze, and implement the algorithm to minimize 8-bit floating point error at inference time for several neural networks. In contrast to traditional sequential summation, our method has significantly lowered numerical errors, achieving classification accuracy on par with high-precision floating-point baselines for multiple image classification tasks. Our dMAC hardware units can reduce power consumption by up to 34.1\% relative to conventional MAC units.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。