提出镜面下降算法优化再生核巴拿赫空间学习问题。
Mirror Descent on Reproducing Kernel Banach Spaces
- 基于对偶空间梯度步的镜面下降法,利用核函数实现迭代优化。
- 在特定条件下实现线性收敛率,约束情形下具标准收敛性。
- 适用于非2-范数的新型再生核巴拿赫空间,适合理论研究者。
机器学习的进展推动了再生核巴拿赫空间(RKBS)作为超越再生核希尔伯特空间(RKHS)更通用框架的关注。已有工作在多种正则化学习框架下建立了表示定理,但该领域尚缺乏有效的优化方法。本文聚焦于配备再生核的巴拿赫空间上的学习问题,提出一种基于镜面下降算法(MDA)的高效优化方法。该方法在巴拿赫空间的对偶空间中执行梯度步骤,利用再生核进行迭代。我们在不同假设下分析算法收敛性,得到两类结果:其一,在特定条件下可实现类似欧氏空间的线性收敛率,并给出证明;其二,在约束设置下展示标准收敛率。为实际应用,我们引入一类新的具有p-范数(p≠2)的RKBS,其具备显式对偶映射和核函数。
原文摘要 · Abstract (English)
Recent advances in machine learning have led to increased interest in reproducing kernel Banach spaces (RKBS) as a more general framework that extends beyond reproducing kernel Hilbert spaces (RKHS). These works have resulted in the formulation of representer theorems under several regularized learning schemes. However, little is known about an optimization method that encompasses these results in this setting. This paper addresses a learning problem on Banach spaces endowed with a reproducing kernel, focusing on efficient optimization within RKBS. To tackle this challenge, we propose an algorithm based on mirror descent (MDA). Our approach involves an iterative method that employs gradient steps in the dual space of the Banach space using the reproducing kernel. We analyze the convergence properties of our algorithm under various assumptions and establish two types of results: first, we identify conditions under which a linear convergence rate is achievable, akin to optimization in the Euclidean setting, and provide a proof of the linear rate; second, we demonstrate a standard convergence rate in a constrained setting. Moreover, to instantiate this algorithm in practice, we introduce a novel family of RKBSs with $p$-norm ($p \neq 2$), characterized by both an explicit dual map and a kernel.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。