首次为非参数核双层优化建立学习理论，给出泛化误差上界。
Learning Theory for Kernel Bilevel Optimization
- 基于再生核希尔伯特空间研究双层优化，实现复杂函数逼近。
- 导出有限样本泛化界，证明梯度法在离散化问题中具统计准确性。
- 适用于需要严格理论保障的机器学习任务，如因果推断。
双层优化已成为解决一类由内层问题最小化解隐式决定外层目标的机器学习问题的有效方法。尽管已有研究主要聚焦于参数化情形，但非参数设置下的学习理论基础仍相对薄弱。本文首次尝试填补这一空白，研究核双层优化（Kernel Bilevel Optimization, KBO），其中内层目标在再生核希尔伯特空间中进行优化。该设定兼具强大的函数逼近能力与严格的理论分析基础。我们利用经验过程理论工具，推导出KBO的新型有限样本泛化界，并进一步评估梯度法在经验离散化问题中的统计精度。我们在一个合成的工具变量回归任务上数值验证了理论结果。
原文摘要 · Abstract (English)
Bilevel optimization has emerged as a technique for addressing a wide range of machine learning problems that involve an outer objective implicitly determined by the minimizer of an inner problem. While prior works have primarily focused on the parametric setting, a learning-theoretic foundation for bilevel optimization in the nonparametric case remains relatively unexplored. In this paper, we take a first step toward bridging this gap by studying Kernel Bilevel Optimization (KBO), where the inner objective is optimized over a reproducing kernel Hilbert space. This setting enables rich function approximation while providing a foundation for rigorous theoretical analysis. In this context, we derive novel finite-sample generalization bounds for KBO, leveraging tools from empirical process theory. These bounds further allow us to assess the statistical accuracy of gradient-based methods applied to the empirical discretization of KBO. We numerically illustrate our theoretical findings on a synthetic instrumental variable regression task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。