用核方法逼近费雪-拉奥梯度流,建立机器学习中的统一理论框架。
Kernel Approximation of Fisher-Rao Gradient Flows
- 基于核方法构建费雪-拉奥梯度流的近似模型。
- 严格证明核近似流的Γ-收敛性,超越点态收敛。
- 连接生成模型、核散度与非参数回归,适合研究生成算法的学者。
本文旨在解决核方法与偏微分方程梯度流交叉领域的若干开放问题。受生成建模和采样进展的启发,我们系统研究了费雪-拉奥与沃瑟斯坦型梯度流的梯度结构、流动方程及其核近似。重点聚焦费雪-拉奥(又称希尔伯特)几何及其多种核基近似,利用偏微分方程梯度流与最优传输理论工具,构建了严谨的理论框架。我们完整刻画了最大均值差异(MMD)空间中的梯度流,揭示其与现有学习与推断算法的联系。分析表明,费雪-拉奥流、斯坦流、核散度与非参数回归之间存在精确理论关联。进一步严格证明了核近似费雪-拉奥流的演化Γ-收敛性,提供超越点态收敛的理论保障。最后通过亥姆霍兹-雷利原理分析能量耗散,建立经典力学理论与现代机器学习实践的重要联系。结果为机器学习中梯度流近似的理解与分析提供了统一的理论基础。
原文摘要 · Abstract (English)
The purpose of this paper is to answer a few open questions in the interface of kernel methods and PDE gradient flows. Motivated by recent advances in machine learning, particularly in generative modeling and sampling, we present a rigorous investigation of Fisher-Rao and Wasserstein type gradient flows concerning their gradient structures, flow equations, and their kernel approximations. Specifically, we focus on the Fisher-Rao (also known as Hellinger) geometry and its various kernel-based approximations, developing a principled theoretical framework using tools from PDE gradient flows and optimal transport theory. We also provide a complete characterization of gradient flows in the maximum-mean discrepancy (MMD) space, with connections to existing learning and inference algorithms. Our analysis reveals precise theoretical insights linking Fisher-Rao flows, Stein flows, kernel discrepancies, and nonparametric regression. We then rigorously prove evolutionary $Γ$-convergence for kernel-approximated Fisher-Rao flows, providing theoretical guarantees beyond pointwise convergence. Finally, we analyze energy dissipation using the Helmholtz-Rayleigh principle, establishing important connections between classical theory in mechanics and modern machine learning practice. Our results provide a unified theoretical foundation for understanding and analyzing approximations of gradient flows in machine learning applications through a rigorous gradient flow and variational method perspective.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。