提出新型投注损失函数，实现[0,1]回归的方差自适应泛化界。
Second-Order Bounds for [0,1]-Valued Regression via Betting Loss
- 设计投注损失，直接建模预测不确定性以获得更优泛化界。
- 新方法在无须预知方差的情况下，实现严格优于一阶界的二阶泛化界。
- 适用于需高精度概率估计的场景，如金融、医疗决策等
我们研究独立同分布设定下的[0,1]取值回归问题。在相关成本敏感分类任务中，Foster等人（2021）证明对数损失最小化器相比平方损失最小化器能获得更优的泛化界——该界依赖于最优分类器的成本，可任意小，此类结果称为一阶界。对于[0,1]回归，我们首先证明对数损失最小化器亦可获得类似一阶界。接着，我们探讨是否存在一种损失函数能实现方差依赖的二阶界（即严格优于一阶界的改进）。我们通过提出一种名为投注损失的新损失函数，正面回答了该问题。该结果具有‘方差自适应’特性：无需任何关于方差的知识即可达成最优界，与显式建模标签方差或标签分布的分布强化学习等方法形成对比。
原文摘要 · Abstract (English)
We consider the $[0,1]$-valued regression problem in the i.i.d. setting. In a related problem called cost-sensitive classification, \citet{foster21efficient} have shown that the log loss minimizer achieves an improved generalization bound compared to that of the squared loss minimizer in the sense that the bound scales with the cost of the best classifier, which can be arbitrarily small depending on the problem at hand. Such a result is often called a first-order bound. For $[0,1]$-valued regression, we first show that the log loss minimizer leads to a similar first-order bound. We then ask if there exists a loss function that achieves a variance-dependent bound (also known as a second order bound), which is a strict improvement upon first-order bounds. We answer this question in the affirmative by proposing a novel loss function called the betting loss. Our result is ``variance-adaptive'' in the sense that the bound is attained \textit{without any knowledge about the variance}, which is in contrast to modeling label (or reward) variance or the label distribution itself explicitly as part of the function class such as distributional reinforcement learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。