自动估算神经网络计算精度，提升模型可靠性。
Algorithms and data structures for automatic precision estimation of neural networks
- 设计算法与数据结构实现浮点计算精度自动估算
- 实验证明几乎所有神经网络都会累积精度损失
- 适合关注模型可信度与结果可解释性的研究者
我们描述了扩展神经网络库以实现浮点计算自动精度估算的算法与数据结构。同时讨论了使估算精确并保持训练与推理高性能的条件。数值实验表明，特定值如推理结果、梯度和与数学预测行为的偏差会因显著精度损失而产生影响。几乎任何神经网络都会累积计算不准确性，导致其实际行为偏离数学模型预测。这说明追踪计算不准确性对推理、训练可靠性和结果可解释性至关重要。
原文摘要 · Abstract (English)
We describe algorithms and data structures to extend a neural network library with automatic precision estimation for floating point computations. We also discuss conditions to make estimations exact and preserve high computation performance of neural networks training and inference. Numerical experiments show the consequences of significant precision loss for particular values such as inference, gradients and deviations from mathematically predicted behavior. It turns out that almost any neural network accumulates computational inaccuracies. As a result, its behavior does not coincide with predicted by the mathematical model of neural network. This shows that tracking of computational inaccuracies is important for reliability of inference, training and interpretability of results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。