arXiv:2509.05106stat.MLcs.LG2025-09被引 3

研究谱算法在输入分布偏移下的收敛性,解决模型不匹配问题。

Spectral Algorithms in Misspecified Regression: Convergence under Covariate Shift

  • 引入重要性权重修正分布偏移,设计加权谱算法
  • 在密度比有界时达到极小极大最优收敛率
  • 提出截断技术应对无界权重,适用于实际场景

本文研究谱算法在协变量偏移下的收敛性质。当源域与目标域输入的边缘分布不同但条件分布不变时,通过引入目标域与源域密度比的重要性权重,构建了再生核希尔伯特空间(RKHS)中的加权谱算法。不同于以往主要关注正确设定情形的研究,本文对更复杂的模型不匹配情形进行了系统性理论分析,即目标函数不在RKHS中。在密度比一致有界条件下,建立了目标函数位于RKHS时的极小极大最优收敛率;针对无界重要性权重的情况,提出新颖的截断技术,在温和正则性条件下实现近最优收敛率,并进一步推广至模型不匹配情形。该工作将经典核学习理论拓展至更贴近实际的应用场景,提供了理解分布偏移与模型不匹配相互作用的系统框架。

原文摘要 · Abstract (English)

This paper investigates the convergence properties of spectral algorithms -- a class of regularization methods originating from inverse problems -- under covariate shift. In this setting, the marginal distributions of inputs differ between source and target domains, while the conditional distribution of outputs given inputs remains unchanged. To address this distributional mismatch, we incorporate importance weights, defined as the ratio of target to source densities, into the learning framework. This leads to a weighted spectral algorithm within a nonparametric regression setting in a reproducing kernel Hilbert space (RKHS). More importantly, in contrast to prior work that largely focuses on the well-specified setting, we provide a comprehensive theoretical analysis of the more challenging misspecified case, in which the target function does not belong to the RKHS. Under the assumption of uniformly bounded density ratios, we establish minimax-optimal convergence rates when the target function lies within the RKHS. For scenarios involving unbounded importance weights, we introduce a novel truncation technique that attains near-optimal convergence rates under mild regularity conditions, and we further extend these results to the misspecified regime. By addressing the intertwined challenges of covariate shift and model misspecification, this work extends classical kernel learning theory to more practical scenarios, providing a systematic framework for understanding their interaction.

谱算法分布偏移核方法收敛分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。