arXiv:2501.10870stat.MLcs.LG2025-01被引 2

用固定带宽高斯核实现鲁棒自适应迁移学习,达到最优收敛速率。

Fixed-Gaussian Spectral Algorithms: Minimax Optimal Rates for Misspecified Learning and Transfer

  • 采用指数衰减正则化,使高斯谱算法对模型误设具普适鲁棒性。
  • 在概念漂移下,迁移学习的额外风险收敛速率逼近最优(对数因子内)。
  • 适合处理非参数回归中模型不匹配与跨数据集迁移问题的研究者。

本文在非参数回归框架下完成两项目标:(1) 确立固定带宽高斯核谱算法在真回归函数属于 Sobolev 空间时的最小最大最优收敛速率;(2) 利用高斯谱算法实现对概念漂移具有鲁棒性和自适应性的迁移学习。尽管已有研究证明了误设谱算法的最小最大最优性,但通常局限于非饱和区域。本文证明,固定带宽高斯核的无限光滑性可提供普遍鲁棒性——只要正则化参数呈指数衰减,任意谱算法即可达到最小最大最优速率,从而将最优性与算法固有资格解耦。基于此,我们主张将高斯谱算法作为鲁棒自适应迁移学习框架的核心组件。具体地,我们推导出该框架的超额风险自适应收敛速率,并表明其最优性仅差对数因子。结果还揭示了概念漂移幅度与样本量对泛化误差的影响。

原文摘要 · Abstract (English)

The principal objective of this work is twofold within nonparametric regression settings: (1) to establish the minimax optimal convergence rates for fixed-bandwidth Gaussian kernel spectral algorithms when the true regression function resides in a Sobolev space, and (2) to apply Gaussian spectral algorithms for achieving robust and adaptive transfer learning under concept shift. While minimax optimality of misspecified spectral algorithms has been established, existing guarantees are typically restricted to the non-saturation regime. We demonstrate that the infinite smoothness of fixed-bandwidth Gaussian kernels provides universal robustness to model misspecification by showing that this kernel choice enables any spectral algorithm to attain minimax optimal rates, provided the regularization parameter decays exponentially. This result effectively decouples optimality from the algorithm's inherent qualification. Building on this, we then advocate Gaussian spectral algorithms as powerful components in a learning framework for robust and adaptive transfer. Specifically, we derive the adaptive convergence rate of the excess risk for this framework and show that the rates are optimal up to logarithmic factors. Our results also reveal the impact of the magnitude of the concept shift and the sample size on the generalization error.

非参数回归迁移学习谱方法最优率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。