提出一种新算法,同时实现高精度预测和强鲁棒性,应对自适应对手的挑战。
Defensive Boosting for Online Probabilistic Forecasting

- 设计防御性增强算法,结合梯度提升与弱学习强化的优势
- 在任意序列上保持与最优线性组合相当的贝叶斯分数表现
- 适用于需要快速、稳定预测的在线场景,如金融或实时监控
我们研究由自适应对手选择的二元结果的在线概率预测问题。给定一个弱假设类 $H$ 的在线学习算法,希望高效获得现有在线提升方法分别提供的两个不可比较的保证:在线梯度提升能在每个序列上与 $H$ 生成的最优预测器竞争贝叶斯分数,但当该最优预测器不存在时无保障;在线弱到强提升可在弱学习条件下使分类错误趋近于零,但条件不满足时表现不佳。本文提出一种简单高效的防御性增强算法(Defensive Booster),同时实现两项保证。在任一自适应序列上,其贝叶斯分数与 $H$ 的跨度中最佳预测器相比,达到与在线梯度提升相同的速率;同时,当实际输出满足平滑弱学习条件时,其贝叶斯分数和随机分类误差也达到与在线分类提升相同速率。这通过操作提升的“双重视角”实现:当随机分类错误持续偏高时,其误判权重构成对所有弱假设均具低优势的平滑重加权,从而提供一个事后硬核证书,证明弱学习条件未成立。此外,还提出强自适应版本,在任意时间区间内都满足双重要求。该算法极为高效:仅需访问一个弱学习器,而先前对比方法需维护大型弱学习器集成。在合成与真实数据流上的实验表明,其预测性能显著优于所有已有基线,且运行时间快数个数量级。
原文摘要 · Abstract (English)
We study online probabilistic forecasting of binary outcomes chosen by an adaptive adversary. Given an online learning algorithm for a weak hypothesis class $H$, we would like to efficiently obtain two incomparable guarantees that existing online boosting techniques provide separately. Online gradient boosting competes in Brier score with the best predictor induced by the span of $H$ on every sequence, but promises nothing when the span does not contain an accurate predictor. Online weak-to-strong boosting drives classification error to zero under a weak-learning condition, but promises little when that condition fails. We give a simple defensive forecasting algorithm, the Defensive Booster, that obtains both guarantees. On every adaptive sequence, its Brier score is competitive with the best prediction induced by the span of $H$ at the same rate as online gradient boosting; simultaneously, whenever the realized transcript satisfies the smooth weak-learning condition, its Brier score and randomized classification error satisfy the same rate guarantee as online classification boosting. This is achieved by operationalizing the "dual view" of boosting: When the algorithm's randomized classification error is persistently high, its mistake weights form a smooth reweighting on which every weak hypothesis has low edge, yielding an ex-post hard-core certificate that the weak-learning condition fails. We also develop a strongly adaptive variant, which satisfies both guarantees on every time interval. The Defensive Booster is very efficient: it accesses just one weak-class learner, whereas the prior online boosting methods we compare against maintain large weak-learner ensembles. Experiments on synthetic and real data streams demonstrate its strong predictive performance (sometimes substantially improving over all prior baselines) coupled with orders-of-magnitude faster runtime.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。