arXiv:2511.23212stat.MLcs.LG2025-11

揭示分位数随机森林变量重要性推断的临界现象

Asymptotic Theory and Phase Transitions for Variable Importance in Quantile Regression Forests

  • 基于针刺损失差构建变量重要性度量,用骑士恒等式处理非光滑损失
  • 发现子采样率β≥1/2时推断失效,估计量收敛到确定性偏移而非正态分布
  • 提出解析偏移校正思路,适合关注高维推断可靠性的研究者

分位数回归森林(QRF)广泛用于非参数条件分位数估计,但变量重要性度量的统计推断因损失函数非光滑性和复杂的偏差-方差权衡而困难。本文建立了基于针刺损失风险差异定义的变量重要性渐近理论。首先,通过骑士恒等式处理不可微的针刺损失,证明了QRF估计量的渐近正态性。其次,揭示由子采样率β(其中s ∝ n^β)主导的“相变”现象:在偏差主导的区域(β ≥ 1/2),对应实践中为最大化预测精度而采用的大子样本量,标准推断失效,估计量收敛至确定性偏差常数而非零均值正态分布。最后,导出了该渐近偏差的显式解析表达式,并讨论了通过解析偏差校正恢复有效推断的理论可行性。结果揭示了预测性能与推断有效性之间的根本权衡,为高维环境下随机森林推断的内在局限提供了理论基础。

原文摘要 · Abstract (English)

Quantile Regression Forests (QRF) are widely used for non-parametric conditional quantile estimation, yet statistical inference for variable importance measures remains challenging due to the non-smoothness of the loss function and the complex bias-variance trade-off. In this paper, we develop a asymptotic theory for variable importance defined as the difference in pinball loss risks. We first establish the asymptotic normality of the QRF estimator by handling the non-differentiable pinball loss via Knight's identity. Second, we uncover a "phase transition" phenomenon governed by the subsampling rate $β$ (where $s \asymp n^β$). We prove that in the bias-dominated regime ($β\ge 1/2$), which corresponds to large subsample sizes typically favored in practice to maximize predictive accuracy, standard inference breaks down as the estimator converges to a deterministic bias constant rather than a zero-mean normal distribution. Finally, we derive the explicit analytic form of this asymptotic bias and discuss the theoretical feasibility of restoring valid inference via analytic bias correction. Our results highlight a fundamental trade-off between predictive performance and inferential validity, providing a theoretical foundation for understanding the intrinsic limitations of random forest inference in high-dimensional settings.

分位数回归变量重要性随机森林渐近理论

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。