用小规模对称群训练模型，实现对大规模对称群的高精度泛化。
Learning the symmetric group: large from small
- 在 $S_{10}$ 上训练变换器，通过一般对换生成词来预测排列。
- 模型在 $S_{25}$ 上达到近 100% 准确率，$S_{16}$ 用相邻对换也表现良好。
- 采用身份增强和分段窗口，提升对变长输入的适应能力。
机器学习可有效探索纯数学难题。其优势在于数学数据无噪声，但挑战在于训练所需数据量大且生成成本高，同时存在模型事后解释困难与深层抽象问题建模难的问题。本文提出一种可扩展方法：在简化任务上训练模型后，使其能泛化至完整任务。具体而言，一个在 $S_{10}$ 上训练的变换器模型，通过一般对换生成的词预测排列，可在 $S_{25}$ 上实现近 100% 准确率；若仅使用相邻对换，则 $S_{10}$ 模型可推广至 $S_{16}$ 保持相似性能。我们引入身份增强以处理变长词，采用分段窗口优化相邻对换训练。最后对比了不同方法变体，并讨论了该方法向其他任务拓展的潜在挑战。
原文摘要 · Abstract (English)
Machine learning explorations can make significant inroads into solving difficult problems in pure mathematics. One advantage of this approach is that mathematical datasets do not suffer from noise, but a challenge is the amount of data required to train these models and that this data can be computationally expensive to generate. Key challenges further comprise difficulty in a posteriori interpretation of statistical models and the implementation of deep and abstract mathematical problems. We propose a method for scalable tasks, by which models trained on simpler versions of a task can then generalize to the full task. Specifically, we demonstrate that a transformer neural-network trained on predicting permutations from words formed by general transpositions in the symmetric group $S_{10}$ can generalize to the symmetric group $S_{25}$ with near 100\% accuracy. We also show that $S_{10}$ generalizes to $S_{16}$ with similar performance if we only use adjacent transpositions. We employ identity augmentation as a key tool to manage variable word lengths, and partitioned windows for training on adjacent transpositions. Finally we compare variations of the method used and discuss potential challenges with extending the method to other tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。