arXiv:2509.10519cs.LG2025-09

改进近似乘法器的梯度估计,让低精度神经网络重训练更准更快。

Gradient Estimation Methods of Approximate Multipliers for High-Accuracy Retraining of Deep Learning Models

  • 用二维/一维查表法精准估算近似乘法器的梯度
  • 在CIFAR-10上提升重训练准确率3.7%~3.8%,ImageNet上提升23.7%
  • 适合做低功耗深度学习加速器的模型优化,尤其适用于视觉Transformer

近似乘法器(AppMult)广泛用于深度学习加速器以降低面积、延迟和功耗。但其引入的算术误差需通过重训练恢复模型精度。关键步骤是计算近似乘积对输入的操作数的梯度。现有方法通常使用精确乘法器(AccMult)的梯度进行估计,导致重训练效果不佳。为此,本文提出两种更精确的梯度估计方法:LUT-2D利用二维查找表精细建模梯度,实现最高重训练精度;LUT-1D则采用一维查找表,结构紧凑且高效,运行时间更短,性能相当。实验表明,在CIFAR-10与卷积神经网络上,LUT-2D和LUT-1D平均提升重训练准确率3.83%和3.72%;在ImageNet与视觉变换器模型上,LUT-1D平均提升23.69%,优于当前最优重训练框架。

原文摘要 · Abstract (English)

Approximate multipliers (AppMults) are widely used in deep learning accelerators to reduce their area, delay, and power consumption. However, AppMults introduce arithmetic errors into deep learning models, necessitating a retraining process to recover accuracy. A key step in retraining is computing the gradient of the AppMult, i.e., the partial derivative of the approximate product with respect to each input operand. Existing approaches typically estimate this gradient using that of the accurate multiplier (AccMult), which can lead to suboptimal retraining results. To address this, we propose two methods to obtain more precise gradients of AppMults. The first, called LUT-2D, characterizes the AppMult gradient with 2-dimensional lookup tables (LUTs), providing fine-grained estimation and achieving the highest retraining accuracy. The second, called LUT-1D, is a compact and more efficient variant that stores gradient values in 1-dimensional LUTs, achieving comparable retraining accuracy with shorter runtime. Experimental results show that on CIFAR-10 with convolutional neural networks, our LUT-2D and LUT-1D methods improve retraining accuracy by 3.83% and 3.72% on average, respectively. On ImageNet with vision transformer models, our LUT-1D method improves retraining accuracy by 23.69% on average, compared to a state-of-the-art retraining framework.

近似计算梯度估计模型重训练加速器优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。