让深度学习训练推理在不同设备上完全一致
RepDL: Bit-level Reproducible Deep Learning Training and Inference
- 通过控制浮点计算顺序和舍入方式实现比特级可复现
- 在多平台测试中确保训练与推理结果完全相同
- 适合需要严格可复现性的科研与工业应用
非确定性和不可复现性是深度学习中的重大挑战，导致不同运行和平台间结果不一致。这些问题主要源于随机数生成和浮点计算。虽然随机性可通过确定性配置控制，但浮点计算的不一致性仍难以解决。为此，我们提出 RepDL，一个开源库，可在多种计算环境中实现深度学习训练与推理的确定性和比特级可复现性。RepDL 通过强制浮点计算的正确舍入和顺序不变性来实现这一目标。源代码已公开于 https://github.com/microsoft/RepDL。
原文摘要 · Abstract (English)
Non-determinism and non-reproducibility present significant challenges in deep learning, leading to inconsistent results across runs and platforms. These issues stem from two origins: random number generation and floating-point computation. While randomness can be controlled through deterministic configurations, floating-point inconsistencies remain largely unresolved. To address this, we introduce RepDL, an open-source library that ensures deterministic and bitwise-reproducible deep learning training and inference across diverse computing environments. RepDL achieves this by enforcing correct rounding and order invariance in floating-point computation. The source code is available at https://github.com/microsoft/RepDL .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。