用Bregman散度统一处理多种损失的校准问题，提升在线学习性能。
Calibeating for general proper losses: A Bregman divergence approach
- 基于Bregman散度构建通用校准框架，统一处理多类合理损失。
- 对Tsallis损失实现对数后悔率，维度依赖更弱，优于以往结果。
- 提出新后悔等式，适用于一般合理损失，对在线学习有理论价值。
本文提出一种基于后悔最小化的通用校准框架。相较于Foster和Hart关于Brier损失（平方损失）和对数损失的专有处理，本工作考虑了包含α-Tsallis损失（α∈[1,2]）和Lipschitz损失在内的广义合理损失族。所研究的Tsallis损失也适用于未缩放版本，可恢复对数损失。分析基于合理损失的Bregman散度视角。技术上，针对所考虑的Tsallis损失族，我们获得了U-校准结果：在该族所有损失上同时实现对数后悔率，且与先前结果相比，维度依赖更弱。此外，我们还得到了一个关于‘成为正则化领导者’的后悔等式，该等式对一般合理损失成立，其推导依赖于两个关于广义方差在线更新公式的结论，其中广义方差是基于Bregman散度的方差推广。
原文摘要 · Abstract (English)
This work introduces a general framework for calibeating based on regret minimization. As compared to Foster and Hart's seminal calibeating work which had specialized treatments of Brier score (squared loss) and log loss, we consider a large family of proper losses that includes $α$-Tsallis losses (for $α\in [1, 2]$) and Lipschitz losses. Our results for Tsallis losses also hold for an unscaled version of Tsallis loss that recovers log loss. Our analysis is oriented around the Bregman divergence view of a proper loss. Technically, our results for the family of Tsallis losses that we consider are U-calibration results, simultaneously obtaining logarithmic regret for all losses in this family while having a weaker dependence on the dimension compared to previous results. Of potential independent interest, we also show a new regret equality for the regret of Be The Regularized Leader. This regret equality holds for general proper losses and itself is based on two results related to online updating formulas for the generalized variance, the latter being a previously introduced generalization of variance based on Bregman divergences.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。