arXiv:2605.07304cs.LG2026-05

用部分顺序关系提升推荐系统个性化效率,减少探索成本。

Latent Order Bandits

  • 基于动作偏好部分顺序构建潜变量模型,放宽对精确分布的依赖
  • 实验显示在相同状态不同评分尺度下,性能优于传统潜变量方法
  • 适合用户偏好结构相似但评价尺度不同的个性化推荐场景

Bandit算法解决各类序列决策问题,但通常在从零开始个性化时样本效率过低。为大幅减少探索时间,潜变量算法利用离散潜状态隐含的跨实例结构,但需准确已知奖励与潜状态的后验分布。然而,获取准确结构模型困难,少量潜状态可能不足以描述所有实例的奖励分布。本文提出潜序带(Latent Order Bandits, LOB),放宽潜变量假设,仅需每状态中动作偏好的部分顺序先验知识。这允许同一状态的不同实例具有不同奖励分布,只要动作偏好顺序一致。例如,流媒体平台上的用户群体可能认同某些电影类型更优,但评分尺度不同。我们给出了适用于全序与偏序的置信上界算法,并给出其后悔上界。为进一步提升实证性能,提出后验采样算法,在多组实验中表明:当同状态实例共享奖励参数时,性能可媲美完整先验潜变量方法;当同状态实例评分尺度不同时,表现更优。

原文摘要 · Abstract (English)

Bandit algorithms solve diverse sequential decision-making problems, but are often too sample-inefficient for from-scratch personalization. To substantially reduce exploration times, latent bandit algorithms exploit cross-instance structure implied by discrete latent states, provided that the posterior distribution of rewards and latent states is known and accurate. However, obtaining an accurate model of this structure is difficult, and a small number of latent states may be insufficient to characterize the reward distributions in all problem instances. We propose latent order bandits (LOB), relaxing the assumptions of latent bandits to require only prior knowledge of a \emph{partial order} of action preferences in each state. This allows instances of the same state to vary in reward distributions, as long as the partial order of actions is shared. For example, groups of users on a streaming service may agree on which movie genres are the best but rate experiences on different scales. We give an upper-confidence bound procedure for the LOB problem, applicable to both total and partial latent orders, and give an upper bound on its regret. To improve empirical performance, we propose a posterior-sampling algorithm and show, in a suite of experiments, that both are competitive with full-prior latent bandits when same-state instances share reward parameters, and preferable to them when reward scales differ between instances with the same latent state.

bandit个性化推荐系统潜变量

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。