arXiv:2512.21208cs.LGmath.DS2025-12

提出学习稳定性剖面,统一分析模型、参数和优化的稳定性机制。

A Learning Stability Profile for Finite-Dimensional Learning Dynamics

  • 构建方向敏感性算子集合,追踪扰动在学习轨迹中的传播。
  • 在正则性与耗散假设下,实现线性化转移算子的均匀或指数衰减绑定。
  • 适用于前馈网络、残差结构、随机梯度等场景,兼顾光滑与非光滑系统。

我们构建了一个有限维敏感性框架,用于研究包含表示、参数和更新变量的学习系统的稳定性。核心是「学习稳定性剖面」——一组方向敏感性算子,记录输入、参数初始化和更新机制的扰动如何沿指定学习轨迹传播。主要结果是一个李雅普诺夫准则:在显式正则性、强制性和耗散性假设下,增量李雅普诺夫能量可导出相关线性化转移算子的统一或指数衰减界。该结果为充分条件,非无条件逆定理。框架区分了终态衰减、剖面有界与次指数增长,避免将非正增长指数等同于一致有界性。剖面被特化至多种标准学习机制:谱界给出前馈网络的前向敏感性估计;耗散性与步长限制给出残差结构的稳定性界;均方收缩假设带来随机梯度方法的参数与更新敏感性界。局部利普希茨系统(包括分段线性网络、邻近映射、投影更新、递归或状态空间迭代)通过克拉克广义雅可比矩阵与变分李雅普诺夫不等式处理。该框架为架构、优化、随机性与非光滑性提供统一的稳定性语言,其作用在于结构性:将已知稳定性机制整合于同一扰动微积分中,并明确每类保证所需假设。

原文摘要 · Abstract (English)

We develop a finite-dimensional sensitivity framework for studying stability in learning systems whose states include representations, parameters, and update variables. The central object is the \emph{Learning Stability Profile}, a collection of directional sensitivity operators that records how perturbations in inputs, parameter initialization, and update mechanisms propagate along a specified learning trajectory. The main result is a Lyapunov criterion for controlling this profile. Under explicit regularity, coercivity, and dissipation assumptions, an incremental Lyapunov energy yields uniform or exponentially decaying bounds on the associated linearized transition operators. The result is stated as a sufficient stability criterion, not as an unconditional converse theorem. The framework also distinguishes terminal decay, profile-wise boundedness, and subexponential growth, avoiding the identification of nonpositive growth exponents with uniform boundedness. The profile is then specialized to several standard learning mechanisms. Spectral bounds give forward sensitivity estimates for feedforward networks. Dissipativity and step-size restrictions give stability bounds for residual architectures. Mean-square contraction assumptions yield parameter and update sensitivity bounds for stochastic gradient methods. Locally Lipschitz systems, including piecewise-linear networks, proximal maps, projected updates, and recurrent or state-space recursions, are handled through Clarke generalized Jacobians and variational Lyapunov inequalities. The resulting framework provides a common stability language for architecture, optimization, stochasticity, and nonsmoothness. Its role is structural: it organizes known stability mechanisms within one perturbation calculus while keeping the hypotheses needed for each guarantee explicit.

稳定性分析学习动态李雅普诺夫非光滑系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。