通过历史数据提升预测模型收敛速度,实现更稳定部署。
Tight Lower Bounds and Improved Convergence in Performative Prediction
- 利用历史重训练数据构建仿射风险最小化算法
- 证明新上界紧致性并突破旧方法收敛下限
- 在多个基准上验证更快收敛,适合动态环境应用
表现性预测框架考虑模型部署后数据分布的变化。确保快速收敛到模型部署后分布不变的稳定点至关重要,尤其在动态环境中。本文扩展了重复风险最小化(RRM)框架,利用先前重训练快照的历史数据,提出一类称为仿射风险最小化的新算法,使更广泛问题可收敛至表现性稳定点。我们为仅使用最终迭代数据的方法提出了新上界,并首次证明该上界及原有上界在相同条件下均为紧致。同时证明,利用历史数据可超越仅用最后迭代数据的RRM下界,且在多个表现性预测基准上实证观察到更快收敛至稳定点。此外,我们首次对仿射风险最小化类中的RRM进行下界分析,量化了其他变体在本框架中可能实现的收敛加速潜力。
原文摘要 · Abstract (English)
Performative prediction is a framework accounting for the shift in the data distribution induced by the prediction of a model deployed in the real world. Ensuring rapid convergence to a stable solution where the data distribution remains the same after the model deployment is crucial, especially in evolving environments. This paper extends the Repeated Risk Minimization (RRM) framework by utilizing historical datasets from previous retraining snapshots, yielding a class of algorithms that we call Affine Risk Minimizers and enabling convergence to a performatively stable point for a broader class of problems. We introduce a new upper bound for methods that use only the final iteration of the dataset and prove for the first time the tightness of both this new bound and the previous existing bounds within the same regime. We also prove that utilizing historical datasets can surpass the lower bound for last iterate RRM, and empirically observe faster convergence to the stable point on various performative prediction benchmarks. We offer at the same time the first lower bound analysis for RRM within the class of Affine Risk Minimizers, quantifying the potential improvements in convergence speed that could be achieved with other variants in our framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。