用杠杆权重改进预测区间,让误差分布不均时更准。
Leverage-Weighted Conformal Prediction
- 基于设计矩阵的杠杆值加权非符合度分数,无需训练额外模型。
- 在异方差场景下实现近似最优条件覆盖,区间宽度几乎不变。
- 适合需要精准置信区间的统计建模与机器学习应用。
分段共形预测可提供有限样本下的无分布预测区间,但生成的区间宽度恒定,在低方差区域过度覆盖,在高方差区域覆盖不足。现有自适应方法需训练辅助模型。本文提出杠杆加权共形预测(LWCP),将非符合度分数按统计杠杆值(帽子矩阵对角线)加权,从设计矩阵几何结构中导出自适应性,而非依赖辅助模型拟合。证明了LWCP对任意权重函数均保持有限样本边际有效性;在异方差通过杠杆体现时,达到渐近最优条件覆盖,且区间宽度几乎无代价;在高斯假设下恢复经典预测区间的形态与宽度,同时保留无分布保证。进一步证明随机杠杆近似可精确保持覆盖性,宽度扰动可控;而原生共形预测存在样本量无关的持续条件覆盖缺口,LWCP可消除该问题。该方法仅需选择权重函数,无其他超参数,计算开销几乎可忽略。合成与真实数据实验验证理论预测,显著降低各类设置下的条件覆盖差异。
原文摘要 · Abstract (English)
Split conformal prediction provides distribution-free prediction intervals with finite-sample marginal coverage, but produces constant-width intervals that overcover in low-variance regions and undercover in high-variance regions. Existing adaptive methods require training auxiliary models. We propose Leverage-Weighted Conformal Prediction (LWCP), which weights nonconformity scores by a function of the statistical leverage -- the diagonal of the hat matrix -- deriving adaptivity from the geometry of the design matrix rather than from auxiliary model fitting. We prove that LWCP preserves finite-sample marginal validity for any weight function; achieves asymptotically optimal conditional coverage at essentially no width cost when heteroscedasticity factors through leverage; and recovers the form and width of classical prediction intervals under Gaussian assumptions while retaining distribution-free guarantees. We further establish that randomized leverage approximations preserve coverage exactly with controlled width perturbation, and that vanilla CP suffers a persistent, sample-size-independent conditional coverage gap that LWCP eliminates. The method requires no hyperparameters beyond the choice of weight function and adds negligible computational overhead to vanilla CP. Experiments on synthetic and real data confirm the theoretical predictions, demonstrating substantial reductions in conditional coverage disparity across settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。