用球面分布高效探索海量动作，解决推荐系统中的可扩展性难题。
Exploring Large Action Sets with Hyperspherical Embeddings using von Mises-Fisher Sampling
- 基于冯·米塞斯-费舍尔分布采样状态向量，再搜索其邻近动作。
- 理论上可等概率探索所有动作，与经典方法效果相当。
- 已成功部署于全球音乐平台推荐系统，支持海量动作探索。
本文提出一种名为冯·米塞斯-费舍尔探索（vMF-exp）的方法，用于在强化学习中高效探索大规模动作集。该方法使用超球面嵌入表示动作，并通过冯·米塞斯-费舍尔分布采样初始状态表示，随后探索其最近邻动作，可扩展至几乎无限数量的候选动作。理论分析表明，在假设条件下，vMF-exp 能渐近保持与经典的玻尔兹曼探索（B-exp）相同的每个动作被探索的概率。而B-exp因需对每个动作计算Softmax值，难以扩展。实验在模拟数据、真实公共数据及全球音乐流媒体平台的大规模推荐系统部署中验证了该方法的有效性。
原文摘要 · Abstract (English)
This paper introduces von Mises-Fisher exploration (vMF-exp), a scalable method for exploring large action sets in reinforcement learning problems where hyperspherical embedding vectors represent these actions. vMF-exp involves initially sampling a state embedding representation using a von Mises-Fisher distribution, then exploring this representation's nearest neighbors, which scales to virtually unlimited numbers of candidate actions. We show that, under theoretical assumptions, vMF-exp asymptotically maintains the same probability of exploring each action as Boltzmann Exploration (B-exp), a popular alternative that, nonetheless, suffers from scalability issues as it requires computing softmax values for each action. Consequently, vMF-exp serves as a scalable alternative to B-exp for exploring large action sets with hyperspherical embeddings. Experiments on simulated data, real-world public data, and the successful large-scale deployment of vMF-exp on the recommender system of a global music streaming service empirically validate the key properties of the proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。