针对数据分布偏移,提出可自适应的深度非参数回归方法。
Adaptive deep nonparametric regression from dependent data under covariate shift
- 用稀疏惩罚神经网络处理协变量偏移与依赖数据
- 在多种时间序列模型下达到近最优收敛速度
- 适合处理真实场景中分布不一致的数据
协变量偏移常出现在实际应用中,源数据与目标数据来自不同分布,此时基于源分布的标准度量不再适用。本文研究在协变量偏移和依赖观测下,基于深度神经网络的非参数分位数与Huber回归估计。考虑满足广义Bernstein型不等式的经典模型,包括独立同分布、ϕ-混合、强混合及C-混合过程。为应对协变量偏移,提出稀疏惩罚深度神经网络(SPDNN)估计器,结合源与目标分布的差异。当密度比未知时,采用两步预训练:第一步构建密度比的最小二乘SPDNN估计;第二步利用该估计进行加权预训练回归估计。对分位数与Huber回归,建立了霍尔德光滑函数类下的非渐近误差界。所提估计器可自适应地达到(至多对数因子)独立同分布数据及多个经典时间序列模型下的极小极大最优收敛率。
原文摘要 · Abstract (English)
Covariate shift often occurs because, in many real applications, the source and the target observations may be generated from different distributions. In this case, the standard metric under the source distribution is not appropriate. This paper considers deep neural network estimators for nonparametric quantile and Huber regression under covariate shift and from dependent observations. We deal with a generalized Bernstein-type inequality that is satisfied by many classical models, including i.i.d. observations, $ϕ$-mixing, strong mixing, and $\mathcal{C}$-mixing processes. To perform the covariate shift phenomenon, we propose a sparse-penalized deep neural network (SPDNN) estimator that takes into account the discrepancy between the source and target distributions of the data. When the density ratio (between the source and target distributions of the covariate) is unknown, a two steps pre-training procedure is carried out: the first step is devoted to the construction of a least squares SPDNN estimator of the density ratio; which is used in the second step to perform a pre-training reweighted SPDNN estimator of the regression function. For both the quantile and the Huber regression, non-asymptotic error bounds of the proposed SPDNN estimators are established in the class of Hölder smooth functions. These estimators can adaptively attain (up to a logarithmic factor) the minimax optimal convergence rate from i.i.d. data as well as from several classical time series models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。