arXiv:2504.00461cs.LGcs.GT2025-04被引 6

首个高效算法,实现有向无环图最短路径的近最优在线学习。

Efficient Near-Optimal Algorithm for Online Shortest Paths in Directed Acyclic Graphs with Bandit Feedback Against Adaptive Adversaries

  • 设计新损失估计与中心点分解,应对自适应对抗者。
  • 达到近最优后悔界 $\tilde O(\sqrt{|E|T\log |X|})$。
  • 适用于博弈、多任务等场景,性能领先。

本文研究在带bandit反馈下,针对自适应对抗者的有向无环图(DAG)在线最短路径问题。给定一个源点 $v_{\mathsf{s}}$ 和汇点 $v_{\mathsf{t}}$ 的 DAG $G = (V, E)$,令 $X \subseteq \{0,1\}^{|E|}$ 表示所有从 $v_{\mathsf{s}}$ 到 $v_{\mathsf{t}}$ 的路径集合。每轮 $t$,我们选择一条路径 $\mathbf{x}_t \in X$,并获得关于损失 $\langle \mathbf{x}_t, \mathbf{y}_t \rangle \in [-1,1]$ 的bandit反馈,其中 $\mathbf{y}_t$ 是对抗性选择的损失向量。目标是使 $T$ 轮后相对于事后最优路径的后悔值最小化。我们提出了首个计算高效的算法,在高概率下实现近极小极大最优后悔界 $\tilde O(\sqrt{|E|T\log |X|})$,其中 $\tilde O(\cdot)$ 隐含了 $|E|$ 的对数因子。该算法通过新颖的损失估计和非平凡的中心点分解实现此界限。作为应用,我们证明该算法在 $m$-集合、广义形式博弈、上校布洛托游戏、有向图最短路径、超立方体以及多任务多臂老虎机等问题中,均提供当前最优的高效算法,并在所有这些设置中实现了改进的高概率后悔保证。

原文摘要 · Abstract (English)

In this paper, we study the online shortest path problem in directed acyclic graphs (DAGs) under bandit feedback against an adaptive adversary. Given a DAG $G = (V, E)$ with a source node $v_{\mathsf{s}}$ and a sink node $v_{\mathsf{t}}$, let $X \subseteq \{0,1\}^{|E|}$ denote the set of all paths from $v_{\mathsf{s}}$ to $v_{\mathsf{t}}$. At each round $t$, we select a path $\mathbf{x}_t \in X$ and receive bandit feedback on our loss $\langle \mathbf{x}_t, \mathbf{y}_t \rangle \in [-1,1]$, where $\mathbf{y}_t$ is an adversarially chosen loss vector. Our goal is to minimize regret with respect to the best path in hindsight over $T$ rounds. We propose the first computationally efficient algorithm to achieve a near-minimax optimal regret bound of $\tilde O(\sqrt{|E|T\log |X|})$ with high probability against any adaptive adversary, where $\tilde O(\cdot)$ hides logarithmic factors in the number of edges $|E|$. Our algorithm leverages a novel loss estimator and a centroid-based decomposition in a nontrivial manner to attain this regret bound. As an application, we show that our algorithm for DAGs provides state-of-the-art efficient algorithms for $m$-sets, extensive-form games, the Colonel Blotto game, shortest walks in directed graphs, hypercubes, and multi-task multi-armed bandits, achieving improved high-probability regret guarantees in all these settings.

在线学习最短路径博弈论带宽反馈

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。