arXiv:2603.19808cs.LGmath.AP2026-03

揭示神经网络群体训练的快慢动态机制,统一解释进化与模型融合方法。

Two-Time-Scale Learning Dynamics: A Population View of Neural Network Training

  • 将参数更新(快)与超参演化(慢)分离建模,引入双时间尺度框架。
  • 证明大群体极限下超参密度服从选择-突变方程,平均行为趋近最优超参。
  • 揭示噪声与多样性在探索与优化间的平衡作用,适合研究训练动力学者。

基于种群的学习范式,如进化策略、基于种群的训练(PBT)及近期的模型融合方法,结合了快速的模型内优化与较慢的种群级适应。尽管这些方法在实践中取得成功,但其集体训练动态仍缺乏通用的数学描述。本文提出一种基于双时间尺度种群动力学的神经网络训练理论框架。将神经网络种群视为交互智能体系统:网络参数通过快速的噪声梯度更新(如SGD/Langevin),而超参数则通过较慢的选择-突变过程演化。我们证明了大规模种群下参数与超参数联合分布的极限,并在强时间尺度分离假设下,推导出超参数密度的选择-突变方程。对每个固定超参数,快速参数动态收敛至玻尔兹曼-吉布斯测度,从而诱导出有效适应度。平均动力学将种群学习与双层优化及经典复制子-突变模型联系起来,给出了种群均值趋向最适超参数的条件,并阐明了噪声与多样性在优化与探索间的平衡作用。数值实验验证了大群体极限与简化后的双时间尺度动态,表明获取有效适应度(显式或通过种群估计)可提升种群级更新性能。

原文摘要 · Abstract (English)

Population-based learning paradigms, including evolutionary strategies, Population-Based Training (PBT), and recent model-merging methods, combine fast within-model optimisation with slower population-level adaptation. Despite their empirical success, a general mathematical description of the resulting collective training dynamics remains incomplete. We introduce a theoretical framework for neural network training based on two-time-scale population dynamics. We model a population of neural networks as an interacting agent system in which network parameters evolve through fast noisy gradient updates of SGD/Langevin type, while hyperparameters evolve through slower selection--mutation dynamics. We prove the large-population limit for the joint distribution of parameters and hyperparameters and, under strong time-scale separation, derive a selection--mutation equation for the hyperparameter density. For each fixed hyperparameter, the fast parameter dynamics relaxes to a Boltzmann--Gibbs measure, inducing an effective fitness for the slow evolution. The averaged dynamics connects population-based learning with bilevel optimisation and classical replicator--mutator models, yields conditions under which the population mean moves toward the fittest hyperparameter, and clarifies the role of noise and diversity in balancing optimisation and exploration. Numerical experiments illustrate both the large-population regime and the reduced two-time-scale dynamics, and indicate that access to the effective fitness, either in closed form or through population-level estimation, can improve population-level updates.

种群学习双时间尺度训练动力学

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。