arXiv:2504.07835cs.LGcs.NA2025-04被引 1

pychop让Python支持自定义低精度计算,加速神经网络与科学计算实验。

pychop: Emulating Low-Precision Arithmetic in Numerical Methods and Neural Networks

  • 在Python中实现可定制浮点格式与多种舍入模式,模拟低精度计算。
  • 在图像分类和目标检测任务中验证低精度对模型性能的影响。
  • 适合研究混合精度算法、量化训练及硬件加速器设计的开发者使用。

为应对计算科学中对低精度算术日益增长的需求,我们利用Python实现低精度计算的模拟——这一语言广泛用于数值分析与机器学习。低精度范式已通过提升计算效率和降低内存占用,革新了深度学习。为促进低精度计算的实验与探索,我们开发了pychop,支持可定制的浮点格式与全面的舍入模式,使用户可在多种应用中快速进行低精度模拟。pychop还提供灵活的数组与张量后端接口,支持在CPU与GPU上高效运行,适用于神经网络部署。本文详述pychop的设计与应用,并基于公开数据集展示了图像分类与目标检测中的低精度模拟结果,揭示了低精度敏感性,提供了关于量化感知训练与后量化影响的宝贵见解。pychop作为混合精度算法的基础工具,推动了科学计算与深度学习部署中数值精度影响的深入研究,助力新型硬件加速器开发。

原文摘要 · Abstract (English)

Motivated by the growing demand for reduced-precision arithmetic in computational science, we exploit lower-precision emulation in Python{--}widely regarded as the dominant programming language for numerical analysis and machine learning. Low-precision paradigms have revolutionized deep learning by enabling more efficient computation and reduced memory footprint while maintaining model fidelity. To better enable numerical experimentation with and exploration of reduced-precision computation, we developed \texttt{pychop}, which supports customizable floating-point formats and a comprehensive set of rounding modes in Python, allowing users to benefit from fast, reduced-precision emulation in numerous applications. \texttt{pychop} also provides flexible interfaces for array and tensor backends, enabling efficient reduced-precision emulation on both CPUs and GPUs for neural network deployment. In this paper, we offer a comprehensive exposition of the design and applications of \texttt{pychop}. Furthermore, we present empirical results on reduced-precision emulation for image classification and object detection using published datasets, illustrating the sensitivity to low precision and delivering valuable insights into its quantization-aware training and post-quantization impacts. Establishing itself as a foundational tool for advancing mixed-precision algorithms, \texttt{pychop} enables in-depth investigations into the effects of numerical precision in scientific computing and deep learning deployment, facilitating the development of novel hardware accelerators.

低精度计算神经网络数值模拟Python工具

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。