用辅助模数提升大规模模加法学习效果，解决训练测试分布偏移问题。
Learning Large-Scale Modular Addition with an Auxiliary Modulus
- 引入辅助模数降低进位频率，保持训练与测试输入分布一致。
- 在大模数、小样本下仍达97%准确率，远超传统方法。
- 适合需要高可扩展性与数据效率的密码学与低资源学习场景。
学习奇偶函数和更一般的模加法是机器学习中的挑战性任务，因其对输入高度敏感。近期研究在求和项数量和模数规模上实现了显著扩展，其核心思想是增加训练序列中的零值，降低有效求和项数以控制训练难度；但此方法导致训练与测试输入分布产生协变量偏移。本文从理论和实验层面分析了该副作用，并提出一种无协变量偏移的模加法学习方法。具体而言，在训练中引入辅助模数 $Kq$，减少进位频率与问题难度，同时保持训练与测试输入分布一致。实验表明，该方法具有强可扩展性和高样本效率：即使在输入长度 $N=64$、模数 $q=974269$ 且数据集较小时（稀疏方法无法学习），本方法仅用10万样本即达到97.0%的 $τ$-准确率（$τ=0.05$），而稀疏方法在相同数据量下仅9.5%，即便扩大至100万样本也仅达93.9%。
原文摘要 · Abstract (English)
Learning parity functions, more general modular addition, is a challenging machine learning task due to its input sensitivity. A recent study substantially scaled modular addition learning in both the number of summands and the modulus. Its key idea is to increase zeros in training sequences, reducing the effective number of summands and thus controlling training difficulty; however, this induces covariate shift between training and test input distributions. This study theoretically and empirically analyzes this side effect and proposes a covariate-shift-free method for modular addition. Specifically, we introduce an auxiliary modulus $Kq$ during training, which reduces wrap-around frequency and problem difficulty while preserving the same input distribution across training and testing. Experiments show strong scalability and sample efficiency: even for large input length $N$, large modulus $q$, and small datasets -- where the sparse method fails to learn -- our method achieves equal or better match accuracy and relaxed $τ$-accuracy. For example, at $N=64$ and $q=974269$, our method trained on 100K samples achieves $97.0\%$ $τ$-accuracy at $τ=0.05$, while the sparse method achieves only $9.5\%$ with the same data size and $93.9\%$ even when extended to 1M samples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。