神经网络在真实高维数据中自发形成跨尺度几何结构。
Scale-Agnostic Kolmogorov-Arnold Geometry in Neural Networks
- 用多尺度分析发现2层MLP在训练中自发生成柯尔莫哥洛夫-阿诺德几何结构。
- 该几何结构在7像素局部到整张28x28图像间保持一致，不随尺度变化。
- 适用于理解模型在真实数据上的隐式几何学习机制，适合研究深层网络泛化者。
Freedman与Mulligan的近期工作表明，浅层多层感知机在合成三维任务训练中会自发产生柯尔莫哥洛夫-阿诺德几何（KAG）结构。然而，这一现象是否存在于真实高维场景及具体空间特性仍不清楚。本文将KAG分析扩展至MNIST手写数字分类任务（784维），采用2层MLP并进行多尺度系统性空间分析。结果发现，训练过程中会涌现出KAG结构，且在不同空间尺度上均稳定存在：从7像素局部邻域到完整的28×28图像。该跨尺度一致性在不同训练策略下均成立，包括标准训练与空间增强训练。这些发现揭示了神经网络在真实高维数据学习过程中会自发构建具有组织性、尺度不变性的几何结构。
原文摘要 · Abstract (English)
Recent work by Freedman and Mulligan demonstrated that shallow multilayer perceptrons spontaneously develop Kolmogorov-Arnold geometric (KAG) structure during training on synthetic three-dimensional tasks. However, it remained unclear whether this phenomenon persists in realistic high-dimensional settings and what spatial properties this geometry exhibits. We extend KAG analysis to MNIST digit classification (784 dimensions) using 2-layer MLPs with systematic spatial analysis at multiple scales. We find that KAG emerges during training and appears consistently across spatial scales, from local 7-pixel neighborhoods to the full 28x28 image. This scale-agnostic property holds across different training procedures: both standard training and training with spatial augmentation produce the same qualitative pattern. These findings reveal that neural networks spontaneously develop organized, scale-invariant geometric structure during learning on realistic high-dimensional data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。