首个可计算的非线性效用多臂老虎机算法,实现理论最优后悔界。
Tractable Multinomial Logit Contextual Bandits with Non-Linear Utilities
- 基于上置信界设计新算法,支持神经网络等非线性效用函数。
- 在真实可实现条件下,达到$ ilde{O}(ar{\sqrt{T}})$后悔率,理论最优。
- 无需神经正切核近似,适用于实际场景中的模型偏差问题。
我们研究用于序列商品组合选择的多项式对数(MNL)上下文老虎机问题。尽管现有研究多假设效用函数与物品特征线性相关,这一假设限制了对物品间复杂交互及用户偏好关系的建模。最近工作(Zhang & Luo, 2024)虽探索了通用效用函数类,但其方法在计算可处理性与统计效率间存在根本权衡。为此,我们提出一种计算高效的算法,结合上置信界原则,专为非线性参数化效用函数(包括神经网络建模)设计。在可实现性假设及效用函数类的温和几何条件下,该算法实现$ ilde{O}(ar{\sqrt{T}})$后悔界,其中$T$为总轮次。结果表明,即使使用神经网络效用,也可在不依赖强假设(如神经正切核近似)的前提下实现尖锐的$ ilde{O}(ar{\sqrt{T}})$后悔率。据我们所知,这是首个在非线性效用下具有计算可处理性且能证明$ ilde{O}(ar{\sqrt{T}})$后悔界的MNL上下文老虎机算法。大量数值实验验证了方法的有效性,显示其在可实现设置及模型误设情形下均具稳健表现。
原文摘要 · Abstract (English)
We study the multinomial logit (MNL) contextual bandit problem for sequential assortment selection. Although most existing research assumes utility functions to be linear in item features, this linearity assumption restricts the modeling of intricate interactions between items and user preferences. A recent work (Zhang & Luo, 2024) has investigated general utility function classes, yet its method faces fundamental trade-offs between computational tractability and statistical efficiency. To address this limitation, we propose a computationally efficient algorithm for MNL contextual bandits leveraging the upper confidence bound principle, specifically designed for non-linear parametric utility functions, including those modeled by neural networks. Under a realizability assumption and a mild geometric condition on the utility function class, our algorithm achieves a regret bound of $\tilde{O}(\sqrt{T})$, where $T$ denotes the total number of rounds. Our result establishes that sharp $\tilde{O}(\sqrt{T})$-regret is attainable even with neural network-based utilities, without relying on strong assumptions such as neural tangent kernel approximations. To the best of our knowledge, our proposed method is the first computationally tractable algorithm for MNL contextual bandits with non-linear utilities that provably attains $\tilde{O}(\sqrt{T})$ regret. Comprehensive numerical experiments validate the effectiveness of our approach, showing robust performance not only in realizable settings but also in scenarios with model misspecification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。