提出零阶牛顿法的严格动力学理论，揭示曲率与方差的权衡机制。
A Formal Kinetic Theory for Zeroth-Order Newton Dynamics:Stein-Corrected Hessian Estimation and Curvature--Variance Trade-offs
- 基于黑箱函数值估计梯度与海森矩阵，引入高斯-斯坦修正消除偏差
- 发现海森噪声通过逆海森夹逼传递，其影响随μ_H^{-4}衰减
- 揭示步长、批量、平滑半径间的曲率-方差权衡，适用于受限查询场景
当梯度和海森矩阵不可用时，零阶牛顿型方法具有实用价值，但其行为与一阶无梯度方法显著不同。本文构建了从黑箱函数值同时估计梯度与海森矩阵的动能框架。朴素的随机方向海森估计器在二次函数上仍存在偏差；需采用高斯-斯坦修正以准确估计高斯平滑目标的海森矩阵。线性化逆海森矩阵揭示两类噪声通道：由逆海森预处理的梯度噪声，以及通过逆海森夹逼传播的海森噪声。在噪声预言机下，第二类噪声携带二阶差分因子μ_H^{-4}。小质量动能提升将有限步牛顿更新映射至欠阻尼相空间模型；过阻尼空间极限导出李雅普诺夫界，暴露步长、批量大小、平滑半径与正则化项之间的曲率-方差权衡。数值实验验证了估计器恒等式、梯度与海森方差定律、维度缩放规律、逆扰动精度，以及在查询预算与正则化消融下的优化行为。
原文摘要 · Abstract (English)
Zeroth-order Newton-type methods are useful when gradients and Hessians are unavailable, but they behave quite differently from first-order gradient-free methods. We develop a kinetic framework for algorithms that estimate both gradient and Hessian from black-box function values. The naive random-direction Hessian estimator turns out to be biased even on quadratics; a Gaussian--Stein correction is needed to estimate the Hessian of the Gaussian-smoothed objective. Linearizing the inverse Hessian exposes two noise channels: gradient noise preconditioned by the inverse Hessian, and Hessian noise transmitted through an inverse-Hessian sandwich. Under a noisy oracle the second channel carries the second-difference factor $μ_H^{-4}$. A small-mass kinetic lift links the finite-step Newton update to an underdamped phase-space model; the overdamped spatial limit yields a Lyapunov bound that exposes the curvature--variance trade-off between step size, batch sizes, smoothing radii, and regularization. Numerical experiments confirm estimator identities, the gradient and Hessian variance laws, dimension scaling, inverse-perturbation accuracy, and optimization behavior under query-budget and regularization ablations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。