对比协作强化学习算法,发现稀疏性控制与探索强度能提升性能。
Comparative Performance of Collaborative Bandit Algorithms: Effect of Sparsity and Exploration Intensity
- 采用软聚类建模用户间关系,允许模糊归属以增强灵活性。
- 控制稀疏性可提高数据效率,探索强度增加能降低错误关系带来的方差。
- 适合研究推荐系统、冷启动问题或强化学习的开发者参考。
本文全面分析协作强化学习算法并进行性能比较。协作上下文老虎机通过在臂(或项目)之间引入关系,实现信息的有效传播,使单个用户(项目)的反馈可在相关用户(项目)间共享,缓解冷启动问题。建模臂间关系主要有两种方法:硬聚类(绝对关系,二值化)与软聚类(模糊隶属)。本文聚焦软聚类,对前沿协作上下文老虎机算法进行广泛实验,研究稀疏性影响及探索强度作为修正机制的作用。数值实验表明,控制协作中的稀疏性可提升数据效率和性能;增加探索强度可有效降低因关系错配导致的方差。进一步发现,引入潜在因子可缓解错配问题,提升带宽参数维度效果更佳。
原文摘要 · Abstract (English)
This paper offers a comprehensive analysis of collaborative bandit algorithms and provides a thorough comparison of their performance. Collaborative bandits aim to improve the performance of contextual bandits by introducing relationships between arms (or items), allowing effective propagation of information. Collaboration among arms allows the feedback obtained through a single user (item) to be shared across related users (items). Introducing collaboration also alleviates the cold user (item) problem, i.e., lack of historical information when a new user (item) arriving to the platform with no prior record of interactions. In the context of modeling the relationships between arms (items), there are two main approaches: Hard and soft clustering. We call approaches that model the relationship between arms in an \textit{absolute} manner as hard clustering, i.e., the relationship is binary. Soft clustering relaxes membership constraints, allowing \textit{fuzzy} assignment. Focusing on the latter, we provide extensive experiments on the state-of-the-art collaborative contextual bandit algorithms and investigate the effect of sparsity and how the exploration intensity acts as a correction mechanism. Our numerical experiments demonstrate that controlling for sparsity in collaboration improves data efficiency and performance as it better informs learning. Meanwhile, increasing the exploration intensity acts as a correction because it effectively reduces variance due to potentially misspecified relationships among users. We observe that this misspecification is further remedied by introducing latent factors, and thus, increasing the dimensionality of the bandit parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。