用z变换分析优化算法收敛性，从梯度下降到随机优化均有突破。
On the Effectiveness of the z-Transform Method in Quadratic Optimization
- 基于z变换研究优化迭代的渐近行为，统一分析框架。
- 在无限维希尔伯特空间中揭示谱维数决定收敛速度。
- 适用于梯度下降、加速方法及随机优化，理论价值高。
z变换是信号处理、控制论、计算机科学和电气工程中的经典工具，可用于通过生成函数研究序列，许多操作在原序列与其z变换间可等价定义。尤其，z变换方法聚焦于渐近行为并支持泰勒展开。本文展示一系列逐步增强显著性与难度的结果，涵盖线性模型与优化算法，证明了z变换方法在推导新渐近结果方面的有效性与通用性。从无限维希尔伯特空间中的最简单梯度下降迭代开始，我们揭示谱维数如何刻画收敛行为；随后将分析扩展至Nesterov加速、平均化技术及随机梯度下降。
原文摘要 · Abstract (English)
The z-transform of a sequence is a classical tool used within signal processing, control theory, computer science, and electrical engineering. It allows for studying sequences from their generating functions, with many operations that can be equivalently defined on the original sequence and its $z$-transform. In particular, the z-transform method focuses on asymptotic behaviors and allows the use of Taylor expansions. We present a sequence of results of increasing significance and difficulty for linear models and optimization algorithms, demonstrating the effectiveness and versatility of the z-transform method in deriving new asymptotic results. Starting from the simplest gradient descent iterations in an infinite-dimensional Hilbert space, we show how the spectral dimension characterizes the convergence behavior. We then extend the analysis to Nesterov acceleration, averaging techniques, and stochastic gradient descent.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。