arXiv:2506.13946stat.MLcs.LG2025-06被引 1

提出非独立同分布下学习率的新分析框架,更贴合真实数据时序依赖。

Rademacher learning rates for iterated random functions

  • 基于迭代随机函数建模时序数据,突破传统独立同分布假设。
  • 推导出依赖数据分布的泛化误差上界,学习率与数据特性相关。
  • 适用于时间序列、马尔可夫链等非平稳场景,适合对现实数据敏感的研究者。

现有监督学习理论多假设训练数据来自独立同分布样本,但许多真实问题存在时间依赖和强相关性,使该假设不切实际。本文考虑由迭代随机函数生成的数据(即非遍历、非周期的时齐马尔可夫链),在控制函数关于第一变量具有压缩性且假设类满足一定正则性条件下,首先建立样本误差的一致收敛性结果。进而证明近似经验风险最小化算法的可学习性,并给出其学习率上界。该上界为数据分布相关,以假设类的Rademacher复杂度表示,能更准确反映数据生成过程的特性。

原文摘要 · Abstract (English)

Most existing literature on supervised machine learning assumes that the training dataset is drawn from an i.i.d. sample. However, many real-world problems exhibit temporal dependence and strong correlations between the marginal distributions of the data-generating process, suggesting that the i.i.d. assumption is often unrealistic. In such cases, models naturally include time-series processes with mixing properties, as well as irreducible and aperiodic ergodic Markov chains. Moreover, the learning rates typically obtained in these settings are independent of the data distribution, which can lead to restrictive choices of hypothesis classes and suboptimal sample complexities for the learning algorithm. In this article, we consider the case where the training dataset is generated by an iterated random function (i.e., an iteratively defined time-homogeneous Markov chain) that is not necessarily irreducible or aperiodic. Under the assumption that the governing function is contractive with respect to its first argument and subject to certain regularity conditions on the hypothesis class, we first establish a uniform convergence result for the corresponding sample error. We then demonstrate the learnability of the approximate empirical risk minimization algorithm and derive its learning rate bound. Both rates are data-distribution dependent, expressed in terms of the Rademacher complexities of the underlying hypothesis class, allowing them to more accurately reflect the properties of the data-generating distribution.

学习率时序数据马尔可夫链泛化误差

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。