用系统理论解释学习系统泛化能力,给出可优化的稳定性证书。
Generalization as a robust performance property of learning-enabled dynamical systems

- 将数据扰动建模为外部干扰,通过积分二次约束描述学习算子增量行为。
- 建立矩阵不等式证书与统一稳定界,分离出单样本敏感度和算法动态增益。
- 适用于梯度下降、加速方法及数据驱动控制,可比较不同学习系统的泛化能力。
通过关注算法稳定性以建立样本外边界,我们为数据驱动优化和反馈控制近似中出现的学习型动态系统提供了泛化性的系统理论解释。针对两个邻近数据集,我们将样本替换建模为作用于灵敏度系统的外部干扰,同时通过积分二次约束编码数据依赖算子的增量行为。基于耗散性论证,我们建立了基于矩阵不等式的证书和统一稳定性界,该界将学习算子的单样本敏感度与算法相关的动态增益分离开来。后者可被优化,从而提供了一种可处理的工具,用于认证和比较学习动态系统的泛化能力。我们的结果恢复了梯度下降的经典结论,自然适用于基于动量的方法(如Heavy-Ball和Nesterov加速),并可扩展至数据驱动控制。
原文摘要 · Abstract (English)
By focusing on algorithmic stability as a means of establishing out-of-sample bounds, we provide a system-theoretic interpretation of generalization in learning-enabled dynamical systems arising in data-driven optimization and feedback control approximation. Given two neighboring datasets, we specifically model sample replacement as an exogenous disturbance acting on a sensitivity system, while the incremental behavior of the data-dependent operator is encoded through an integral quadratic constraint. By relying on dissipativity arguments, we establish a matrix inequality-based certificate and a uniform stability bound that separates the one-sample sensitivity of the learned operator, and an algorithm-dependent dynamical gain. The latter can then be optimized, offering a tractable tool for certifying and comparing generalization capabilities of learning dynamics. We show that our results recover classical ones for gradient descent, apply naturally to momentum-based methods such as heavy-ball and Nesterov acceleration, and extend to data-driven control.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。