根据误差传播规律，动态调整神经网络推理的精度，提升效率与准确率平衡。
Mixed precision accumulation for neural network inference guided by componentwise forward error analysis
- 基于逐分量前向误差分析，识别输出各部分误差敏感度
- 实验显示相比统一精度，可显著优化计算成本与精度权衡
- 适合对推理效率有要求的部署场景，如边缘设备
本文提出一种数学严谨的混合精度累加策略，用于神经网络推理。该策略基于新的逐分量前向误差分析，揭示线性层输出各分量的误差与权重-输入内积的条件数、激活函数条件数的乘积成正比。由于这些条件数在不同分量间差异显著，为引入混合精度提供了机会：每个分量应使用与其条件数乘积成反比的精度进行累加。我们设计了一种数值算法：先以低精度计算所有分量，利用结果估计条件数，再仅对条件数大的分量用高精度重算。在多个网络和数据集上的测试表明，该方法相比统一精度基线能显著改善成本-精度权衡。
原文摘要 · Abstract (English)
This work proposes a mathematically founded mixed precision accumulation strategy for the inference of neural networks. Our strategy is based on a new componentwise forward error analysis that explains the propagation of errors in the forward pass of neural networks. Specifically, our analysis shows that the error in each component of the output of a linear layer is proportional to the condition number of the inner product between the weights and the input, multiplied by the condition number of the activation function. These condition numbers can vary widely from one component to the other, thus creating a significant opportunity to introduce mixed precision: each component should be accumulated in a precision inversely proportional to the product of these condition numbers. We propose a numerical algorithm that exploits this observation: it first computes all components in low precision, uses this output to estimate the condition numbers, and recomputes in higher precision only the components associated with large condition numbers. We test our algorithm on various networks and datasets and confirm experimentally that it can significantly improve the cost--accuracy tradeoff compared with uniform precision accumulation baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。