用Transformer生成高质量伪随机数，理论证明可行且实验通过测试
Transformers in Pseudo-Random Number Generation: A Dual Perspective on Theory and Practice
- 用自回归Transformer模拟线性同余和梅森旋转算法
- 生成的随机数通过多数NIST测试，热力图显示统计随机性
- 适合对随机性要求高的模型训练与安全应用
伪随机数生成器（PRNG）是高度非线性的过程，也是大语言模型优化的关键组件。Transformer在处理复杂非线性关系方面表现优异，因此基于Transformer生成高质量伪随机数具有合理性。本文从理论与实践双重视角探索该问题，理论上证明仅解码器结构的Transformer配合思维链可模拟线性同余生成器（LCG）与梅森旋转生成器（MT）。基于此，得出结论：对数精度的解码器仅用的Transformer可表示非均匀AC⁰。理论成果经实验验证：基于Transformer的PRNG生成的随机数通过了多数NIST测试，其热力图呈现明显统计随机性。最后评估了其在预测攻击下的抗性。
原文摘要 · Abstract (English)
Pseudo-random number generators (PRNGs) are high-nonlinear processes, and they are key blocks in optimization of Large language models. Transformers excel at processing complex nonlinear relationships. Thus it is reasonable to generate high-quality pseudo-random numbers based on transformers. In this paper, we explore this question from both theoretical and practical perspectives, highlighting the potential benefits and implications of Transformer in PRNGs. We theoretically demonstrate that decoder-only Transformer models with Chain-of-Thought can simulate both the Linear Congruential Generator (LCG) and Mersenne Twister (MT) PRNGs. Based on this, we conclude that the log-precision decoder-only Transformer can represent non-uniform $\text{AC}^0$. Our simulative theoretical findings are validated through experiments. The random numbers generated by Transformer-based PRNGs successfully pass the majority of NIST tests, whose heat maps exhibit clear statistical randomness. Finally, we assess their capability in prediction attacks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。