提出一种去偏方法,让非参数回归估计达到√n一致性和正态性。
Debiased Regression for Root-N-Consistent Conditional Mean Estimation
- 通过添加偏差校正项,改进非参数回归估计器。
- 在弱收敛率条件下实现√n一致性与渐近正态性。
- 无需模型假设,适合高维和机器学习场景的置信区间构建。
本文提出一种针对回归估计器(包括高维与非参数回归)的去偏方法。非参数回归虽能数据驱动地估计回归函数且假设少,但通常无法达到√n一致性的收敛速度,许多方法(如机器学习中的)也缺乏渐近正态性保证。为此,我们通过为原估计器添加偏差校正项,扩展了半参数分析中的经典一步估计法。对每个数据点,估计原非参数估计器的条件残差期望(例如用核(Nadaraya-Watson)回归计算),并将其作为偏差减少项。理论分析表明,在原估计器与条件残差估计器满足温和收敛率条件时,所提估计器可实现√n一致性与渐近正态性。该方法保持模型无关性,只要满足收敛率条件即可。优势包括更高的估计精度和更简便的置信区间构造。
原文摘要 · Abstract (English)
This study introduces a debiasing method for regression estimators, including high-dimensional and nonparametric regression estimators. For example, nonparametric regression methods allow for the estimation of regression functions in a data-driven manner with minimal assumptions; however, these methods typically fail to achieve $\sqrt{n}$-consistency in their convergence rates, and many, including those in machine learning, lack guarantees that their estimators asymptotically follow a normal distribution. To address these challenges, we propose a debiasing technique for nonparametric estimators by adding a bias-correction term to the original estimators, extending the conventional one-step estimator used in semiparametric analysis. Specifically, for each data point, we estimate the conditional expected residual of the original nonparametric estimator, which can, for instance, be computed using kernel (Nadaraya-Watson) regression, and incorporate it as a bias-reduction term. Our theoretical analysis demonstrates that the proposed estimator achieves $\sqrt{n}$-consistency and asymptotic normality under a mild convergence rate condition for both the original nonparametric estimator and the conditional expected residual estimator. Notably, this approach remains model-free as long as the original estimator and the conditional expected residual estimator satisfy the convergence rate condition. The proposed method offers several advantages, including improved estimation accuracy and simplified construction of confidence intervals.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。