为多项式对数模型设计高效实验,兼顾精度与计算速度。
Optimal Design for Multinomial Logit Model with Applications to Best Assortment Identification
- 将复杂选择问题转化为可解的整数规划,实现精准设计。
- 在1000个商品中仅需约20次测试即可识别最优组合。
- 适合做推荐系统优化或个性化商品排序的研究者参考。
我们研究了多项式对数(MNL)强化学习中的最优实验设计问题,其中智能体从总数为N的物品集合中反复选择一个大小为K的子集,并观察单选反馈。与线性或广义线性强化学习不同,MNL强化学习具有组合型动作空间,使得经典最优设计方法和对所有子集的朴素优化在计算上不可行。为此,我们提出一种计算高效的最优设计框架,通过两种互补策略同时实现统计效率与可扩展性:(i) 将设计最优解重构为带有求解器认证提前停止的0-1混合整数线性规划(MILP);(ii) 提出全多项式时间的提升型设计方法,用可处理的代理目标替代非线性目标函数。基于Kiefer-Wolfowitz等价定理,我们建立了近似G-最优性保证,并刻画了由此产生的统计-计算权衡。作为应用,我们开发了一种适用于线性效用和非均匀收益的MNL强化学习最优组合识别算法,证明其实例依赖的样本复杂度为$ ilde{O}ig(rac{d \ ext{log} N}{Δ^2}ig)$,其中$d$为特征维度,$N$为臂的数量,$Δ$为最小收益差距。
原文摘要 · Abstract (English)
We study optimal experimental design for multinomial logit (MNL) bandits, where an agent repeatedly selects a subset of $K$ items from a ground set of size $N$ and observes single-choice feedback. Unlike linear or generalized linear bandits, MNL bandits have a combinatorial action space, which makes classical optimal design approaches and naive optimization over all subsets computationally intractable. We propose a computationally efficient optimal design framework for MNL models that achieves both statistical efficiency and scalability through two complementary approaches: (i) an exact or certified-approximate reformulation of the design oracle as a $0$-$1$ mixed-integer linear program (MILP) with solver-certified early stopping, and (ii) a fully polynomial-time lifted design that replaces the nonlinear objective with a tractable surrogate. Using the Kiefer-Wolfowitz equivalence theorem, we establish near G-optimality guarantees and characterize the induced statistical-computational trade-offs. As an application, we develop a best assortment identification algorithm for MNL bandits with linear utilities and non-uniform revenues, and prove an instance-dependent sample complexity of $\tilde{O}\big(\frac{d \log N}{Δ^2}\big)$, where $d$ is the feature dimension, $N$ is the number of arms, and $Δ$ is the minimum revenue gap.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。