测试发现主流框架的随机数生成器存在统计缺陷，影响模型复现性。
Statistical Quality and Reproducibility of Pseudorandom Number Generators in Machine Learning technologies
- 用TestU01大碾压测试对比框架内与原生实现的随机数质量
- 896条独立随机流中，标称抗压的PCG/Philox仍部分失败
- 框架版本与原版算法差异导致测试表现不同，提示实现细节关键
机器学习框架依赖伪随机数生成器（PRNG）完成数据打乱、权重初始化、丢弃层和优化等任务。然而，这些生成器在PyTorch、TensorFlow和NumPy等框架中的统计质量与可复现性尚未得到充分研究。本文使用严格的TestU01 BigCrush测试套件，对机器学习框架中使用的三种生成器（梅森旋转、PCG、Philox）及其原始C实现进行对比评估，每种生成器测试896条独立随机流。结果挑战了其“抗压”宣称，发现即便被标记为“crush-resistant”的生成器（如PCG、Philox）仍可能在某些统计测试中失败。更意外的是，同一算法的原生版本与框架集成版本在失败模式上存在差异，揭示了潜在的实现差异。
原文摘要 · Abstract (English)
Machine learning (ML) frameworks rely heavily on pseudorandom number generators (PRNGs) for tasks such as data shuffling, weight initialization, dropout, and optimization. Yet, the statistical quality and reproducibility of these generators-particularly when integrated into frameworks like PyTorch, TensorFlow, and NumPy-are underexplored. In this paper, we compare the statistical quality of PRNGs used in ML frameworks (Mersenne Twister, PCG, and Philox) against their original C implementations. Using the rigorous TestU01 BigCrush test suite, we evaluate 896 independent random streams for each generator. Our findings challenge claims of statistical robustness, revealing that even generators labeled ''crush-resistant'' (e.g., PCG, Philox) may fail certain statistical tests. Surprisingly, we can observe some differences in failure profiles between the native and framework-integrated versions of the same algorithm, highlighting some implementation differences that may exist.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。