改用二维网格表示，神经网络可零误差算大数乘法，颠覆了长程依赖的误解。
On the Mirage of Long-Range Dependency, with an Application to Integer Multiplication
- 将二进制数排成2D网格，乘法变为局部3×3操作
- 仅321参数的神经元自动机实现683倍长度外推完美计算
- 提醒研究者警惕“长程依赖”归因，可能源于计算空间设计
整数乘法长期被视为神经网络难题，普遍归因于进位链带来的O(n)长程依赖。我们指出这一诊断错误：长程依赖并非乘法本身属性，而是计算时空选择造成的幻象。通过将两个n比特二进制数排列为2D外积网格，长乘法每一步均可简化为3×3局部邻域操作。在此表示下，仅含321个可学习参数的神经元自动机实现了高达训练范围683倍的完美长度泛化。五种替代架构（包括6,625参数Transformer、Transformer+RoPE和Mamba）在相同表示下均失败。我们进一步分析部分成功如何导致社区误判，并主张所有被归因为长程依赖的任务，应首先检验依赖是否源于任务本质或计算时空构造。
原文摘要 · Abstract (English)
Integer multiplication has long been considered a hard problem for neural networks, with the difficulty widely attributed to the O(n) long-range dependency induced by carry chains. We argue that this diagnosis is wrong: long-range dependency is not an intrinsic property of multiplication, but a mirage produced by the choice of computational spacetime. We formalize the notion of mirage and provide a constructive proof: when two n-bit binary integers are laid out as a 2D outer-product grid, every step of long multiplication collapses into a $3 \times 3$ local neighborhood operation. Under this representation, a neural cellular automaton with only 321 learnable parameters achieves perfect length generalization up to $683\times$ the training range. Five alternative architectures -- including Transformer (6,625 params), Transformer+RoPE, and Mamba -- all fail under the same representation. We further analyze how partial successes locked the community into an incorrect diagnosis, and argue that any task diagnosed as requiring long-range dependency should first be examined for whether the dependency is intrinsic to the task or induced by the computational spacetime.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。