突破线性带宽的内积限制,实现最优传输场景下的高效学习。
Linear Bandits beyond Inner Product Spaces, the case of Bandit Optimal Transport
- 将动作集嵌入希尔伯特子空间,用最小二乘构建置信集。
- 在最优传输问题中实现近似最优的后悔上界,随代价函数光滑度在√T与T之间变化。
- 适用于高维、非内积结构的在线优化任务,如推荐系统与临床试验。
线性带宽长期是在线学习的核心课题,广泛应用于推荐系统与自适应临床试验。传统方法依赖代价参数与决策变量之间的内积结构来最小化损失,但该结构并非必要条件,且无法涵盖最优传输等问题。本文以Kantorovich型最优传输为例,证明即使无内积结构,仍可实现高效的线性带宽学习。提出对经典OFUL算法的改进:将动作集嵌入希尔伯特子空间,通过最小二乘法构造置信集,并通过惩罚机制约束乐观性。结合罚项(熵正则)运输收敛结果,完成分析。在近似误差范围内,该算法达到与OFUL相同的轨迹后悔上界,利用函数回归技术转化为最坏情况后悔界。其后悔率在~O(√T)与O(T)间插值,取决于代价函数的正则性;在有限维情形下恢复参数化速率~O(√(dT))。
原文摘要 · Abstract (English)
Linear bandits have long been a central topic in online learning, with applications ranging from recommendation systems to adaptive clinical trials. Their general learnability has been established when the objective is to minimise the inner product between a cost parameter and the decision variable. While this is highly general, this reliance on an inner product structure belies the name of \emph{linear} bandits, and fails to account for problems such as Optimal Transport. Using the Kantorovich formulation of Optimal Transport as an example, we show that an inner product structure is \emph{not} necessary to achieve efficient learning in linear bandits. We propose a refinement of the classical OFUL algorithm that operates by embedding the action set into a Hilbertian subspace, where confidence sets can be built via least-squares estimation. Actions are then constrained to this subspace by penalising optimism. The analysis is completed by leveraging convergence results from penalised (entropic) transport to the Kantorovich problem. Up to this approximation term, the resulting algorithm achieves the same trajectorial regret upper bounds as the OFUL algorithm, which we turn into worst-case regret using functional regression techniques. Its regret interpolates between $\tilde{\mathcal O}(\sqrt{T})$ and ${\mathcal O}(T)$, depending on the regularity of the cost function, and recovers the parametric rate $\tilde{\mathcal O}(\sqrt{dT})$ in finite-dimensional settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。