解决在线结构化预测中反馈不完整的问题,提升实际应用可行性。
Bandit and Delayed Feedback in Online Structured Prediction
- 设计新算法应对带宽和延迟反馈,降低对完整输出信息的依赖。
- 提出独立于输出规模K的O(T^{2/3}) regret上界,突破传统方法限制。
- 适用于需要实时决策且反馈受限的场景,如推荐系统与自动标注。
在线结构化预测是基于输入和历史观测,依次预测具有复杂结构的输出的任务,涵盖在线分类。近期研究在全信息反馈设定下实现了代理损失遗憾(surrogate regret)的有限上界,即相对于最优代理损失的额外损失。然而,实践中全信息反馈往往不现实,因为它要求立即获取复杂输出的完整结构。为此,本文提出适用于更弱反馈(带宽反馈与延迟反馈)的算法。对于带宽反馈,使用标准逆权重梯度估计器,获得关于时间跨度T和输出集大小K的O(√(KT))代理遗憾上界。当输出结构高度复杂时,K极大,导致上界不可接受。为此,我们提出另一算法,通过精心设计的伪逆矩阵估计器,实现与K无关的O(T^{2/3})代理遗憾上界。此外,我们数值比较了所提算法与现有方法的性能。针对延迟反馈,本文提供了覆盖多种场景(全信息、带宽反馈,固定与可变延迟)的算法及遗憾分析。
原文摘要 · Abstract (English)
Online structured prediction is a task of sequentially predicting outputs with complex structures based on inputs and past observations, encompassing online classification. Recent studies showed that in the full-information setting, we can achieve finite bounds on the \textit{surrogate regret}, \textit{i.e.,}~the extra target loss relative to the best possible surrogate loss. In practice, however, full-information feedback is often unrealistic as it requires immediate access to the whole structure of complex outputs. Motivated by this, we propose algorithms that work with less demanding feedback, \textit{bandit} and \textit{delayed} feedback. For bandit feedback, by using a standard inverse-weighted gradient estimator, we achieve a surrogate regret bound of $O(\sqrt{KT})$ for the time horizon $T$ and the size of the output set $K$. However, $K$ can be extremely large when outputs are highly complex, resulting in an undesirable bound. To address this issue, we propose another algorithm that achieves a surrogate regret bound of $O(T^{2/3})$, which is independent of $K$. This is achieved with a carefully designed pseudo-inverse matrix estimator. Furthermore, we numerically compare the performance of these algorithms, as well as existing ones. Regarding delayed feedback, we provide algorithms and regret analyses that cover various scenarios, including full-information and bandit feedback, as well as fixed and variable delays.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。