arXiv:2508.00523cs.LG2025-08AAAI被引 2

提出新算法,解决带延迟反馈的非子模优化问题。

Online Nonsubmodular Optimization with Delayed Feedback in the Bandit Setting

  • 用单点梯度估计和累积梯度更新,降低延迟影响。
  • 新界达平均延迟,最坏情况下优于旧方法。
  • 适合延迟不均的在线学习场景,如结构化稀疏学习。

研究带延迟反馈的在线非子模优化问题,损失函数为α-弱DR-子模且β-弱DR-超模。现有工作在最大延迟d下达到$ \mathcal{O}(nd^{1/3}T^{2/3})$的$(α,β)$-遗憾界,但对异常延迟敏感,且将延迟与带宽反馈效应耦合。本文提出DBGD-NF算法,利用所有可用梯度更新,实现关于平均延迟$ \bar{d} = \frac{1}{T}\sum_{t=1}^T d_t \leq d$的$ \mathcal{O}(n\bar{d}^{1/3}T^{2/3})$遗憾界。进一步引入阻塞更新机制,解耦延迟与反馈影响,获得$ \mathcal{O}(n(T^{2/3} + \sqrt{dT}))$遗憾界。当$d = \mathcal{O}(T^{1/3})$时,恢复无延迟情形的$ \mathcal{O}(nT^{2/3})$性能。实验验证了在结构化稀疏学习中的优越性。

原文摘要 · Abstract (English)

We investigate the online nonsubmodular optimization with delayed feedback in the bandit setting, where the loss function is $α$-weakly DR-submodular and $β$-weakly DR-supermodular. Previous work has established an $(α,β)$-regret bound of $\mathcal{O}(nd^{1/3}T^{2/3})$, where $n$ is the dimensionality and $d$ is the maximum delay. However, its regret bound relies on the maximum delay and is thus sensitive to irregular delays. Additionally, it couples the effects of delays and bandit feedback as its bound is the product of the delay term and the $\mathcal{O}(nT^{2/3})$ regret bound in the bandit setting without delayed feedback. In this paper, we develop two algorithms to address these limitations, respectively. Firstly, we propose a novel method, namely DBGD-NF, which employs the one-point gradient estimator and utilizes all the available estimated gradients in each round to update the decision. It achieves a better $\mathcal{O}(n\bar{d}^{1/3}T^{2/3})$ regret bound, which is relevant to the average delay $\bar{d} = \frac{1}{T}\sum_{t=1}^T d_t\leq d$. Secondly, we extend DBGD-NF by employing a blocking update mechanism to decouple the joint effect of the delays and bandit feedback, which enjoys an $\mathcal{O}(n(T^{2/3} + \sqrt{dT}))$ regret bound. When $d = \mathcal{O}(T^{1/3})$, our regret bound matches the $\mathcal{O}(nT^{2/3})$ bound in the bandit setting without delayed feedback. Compared to our first $\mathcal{O}(n\bar{d}^{1/3}T^{2/3})$ bound, it is more advantageous when the maximum delay $d = o(\bar{d}^{2/3}T^{1/3})$. Finally, we conduct experiments on structured sparse learning to demonstrate the superiority of our methods.

在线优化延迟反馈非子模带宽学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。