arXiv:2602.09457stat.MLcs.DS2026-02

提出随机顺序下小损失后悔界新方法,适用于聚类与低秩逼近等广泛问题。

From Average Sensitivity to Small-Loss Regret Bounds under Random-Order Model

  • 基于离线算法的近似性与敏感度控制,构建在线学习后悔界
  • 实现阶为 $\tilde O(φ^\star(\mathrm{OPT}_T))$ 的小损失后悔界
  • 无需损失函数结构假设,适用于聚类、回归等场景

研究随机顺序模型下的在线学习,其中损失函数集合由对抗方选定但以均匀随机顺序呈现。通过扩展Dong和Yoshida(2023)的批处理到在线转换方法,我们证明:若一个离线算法具有 $(1+\varepsilon)$-近似保证、平均敏感度受函数 $φ(\varepsilon)$ 控制,并对 $\varepsilon$ 稳定,则可获得典型阶为 $\tilde O(φ^{\star}(\mathrm{OPT}_T))$ 的小损失后悔界,其中 $φ^{\star}$ 为 $φ$ 的凹共轭,$\mathrm{OPT}_T$ 为 $T$ 轮内的离线最优值,$\tilde O$ 隐含关于 $T$ 的多项式对数因子。该结果改进了原始的 $(1+\varepsilon)$-近似后悔界,适用于包括在线 $k$-均值聚类和在线低秩逼近在内的广泛问题。进一步将该方法应用于在线子模函数最小化,利用子模超图的 $(1\pm\varepsilon)$-割稀疏化,得到 $\tilde O(n^3 + n^{3/4}\mathrm{OPT}_T^{3/4})$ 的小损失后悔界,其中 $n$ 为基集大小;同时证明其在在线 $\ell_1$ 回归中的适用性。本工作揭示了稀疏化及相关算法技术在随机顺序模型中实现小损失后悔界的能力,且无需对损失函数施加如线性或光滑性等结构性假设。

原文摘要 · Abstract (English)

We study online learning in the random-order model, where the multiset of loss functions is chosen adversarially but revealed in a uniformly random order. By extending the batch-to-online transformation of Dong and Yoshida (2023), we show that if an offline algorithm enjoys a $(1+\varepsilon)$-approximation guarantee, an average sensitivity bound controlled by a function $φ(\varepsilon)$, and stability with respect to $\varepsilon$, then we can obtain a small-loss regret bound typically of order $\tilde O(φ^{\star}(\mathrm{OPT}_T))$, where $φ^{\star}$ is the concave conjugate of $φ$, $\mathrm{OPT}_T$ is the offline optimum over $T$ rounds, and $\tilde O$ hides polylogarithmic factors in $T$. Our result refines their original $(1+\varepsilon)$-approximate regret guarantee and applies to a broad class of problems, including online $k$-means clustering and online low-rank approximation. We further apply our approach to online submodular function minimization using $(1\pm\varepsilon)$-cut sparsifiers of submodular hypergraphs, obtaining a small-loss regret bound of $\tilde O(n^3 + n^{3/4}\mathrm{OPT}_T^{3/4})$, where $n$ is the ground-set size; we also demonstrate its applicability to online $\ell_1$ regression. Our work sheds light on the power of sparsification and related algorithmic techniques in achieving small-loss regret bounds in the random-order model, without requiring structural assumptions on loss functions, such as linearity or smoothness.

在线学习后悔界随机顺序稀疏化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。