从内部传播几何看深度学习,发现模型性能不止取决于输出结果。
The Propagation Field: A Geometric Substrate Theory of Deep Learning
- 将神经网络视为隐藏状态轨迹与雅可比算子的几何场,而非仅关注输入输出
- 相同输出表现的模型,其内部传播结构可差上百倍,影响泛化与鲁棒性
- 适用于需跨路径泛化、持续学习或对模型内部机制敏感的研究场景
现代深度学习将神经网络理解为从输入到输出的端点函数。受物理学从力到几何转变的启发,我们提出应通过网络内部传播的几何结构来理解其本质。定义神经传播场为深度方向上隐藏状态轨迹与局部雅可比算子的集合。端点损失仅约束该场的边界行为,导致其内部几何未被充分确定。我们发现,端点等价的模型在轨迹与雅可比结构上可相差数个数量级。引入路径敏感性、求解器一致性、轨迹/雅可比保留等可观测场度量。在受控教师流与偏微分方程系统中,仅端点拟合无法恢复底层传播规律。在真实多路径任务中,与观测结构对齐的场感知目标能提升未见路径泛化、分布外鲁棒性与校准能力,但过度约束会导致崩溃。在持续学习中,场保持正则化补充了回放与蒸馏:在Split CIFAR-100上,结合场保持的DER++提升了平均准确率、反向迁移与场保留指标。这些结果表明,传播场质量是超越端点性能可度量、可训练的神经网络属性。
原文摘要 · Abstract (English)
Modern deep learning treats neural networks primarily as endpoint functions from inputs to outputs. Inspired by the shift from force to geometry in physics, we ask whether a network should instead be understood through the geometry of its internal propagation. We define a neural propagation field as the collection of hidden-state trajectories and local Jacobian operators across depth. Endpoint losses constrain only the boundary behavior of this field, leaving its interior geometry underdetermined. We show that endpoint-equivalent models can differ by orders of magnitude in trajectory and Jacobian structure, and introduce observable field metrics such as path sensitivity, solver consistency, and trajectory/Jacobian retention. In controlled teacher-flow and PDE systems, endpoint fitting fails to recover the underlying propagation law. In real multi-path tasks, field-aware objectives improve unseen-path generalization, OOD robustness, and calibration when aligned with the observation structure, but can collapse when over-constrained. In continual learning, field-preservation regularization complements replay and distillation: on Split CIFAR-100, DER++ with field preservation improves average accuracy, backward transfer, and field-retention metrics. These results identify propagation-field quality as a measurable and trainable property of neural networks beyond endpoint performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。