用神经网络改进推荐系统中的探索与利用平衡策略。
Neural Risk-sensitive Satisficing in Contextual Bandits
- 用神经网络替代线性近似，捕捉特征与收益间的非线性关系。
- 在多个真实数据集上表现优于传统方法，尤其在冷启动场景下提升明显。
- 适合需要动态适应复杂环境的个性化推荐系统开发者。
上下文老虎机问题是一种强化学习任务，广泛应用于推荐系统中，以满足实时需求、实现个性化推荐并缓解冷启动问题。然而，现有算法面临状态-动作空间大、学习成本高、探索与利用难以平衡，以及性能受应用场景影响大等挑战。为此，Tsuboya 等人提出区域线性风险敏感满意算法（RegLinRS），根据代理是否达到目标来切换探索与利用。但该方法基于特征对奖励期望进行线性近似，限制了其在非线性关系环境中的适用性。为应对更复杂的场景，本文提出神经风险敏感满意算法（NeuralRS），将神经网络引入 RegLinRS，显著提升了算法在非线性环境中的表现，并在多个真实数据集上验证了其有效性。
原文摘要 · Abstract (English)
The contextual bandit problem, which is a type of reinforcement learning tasks, provides an effective framework for solving challenges in recommendation systems, such as satisfying real-time requirements, enabling personalization, addressing cold-start problems. However, contextual bandit algorithms face challenges since they need to handle large state-action spaces sequentially. These challenges include the high costs for learning and balancing exploration and exploitation, as well as large variations in performance that depend on the domain of application. To address these challenges, Tsuboya et~al. proposed the Regional Linear Risk-sensitive Satisficing (RegLinRS) algorithm. RegLinRS switches between exploration and exploitation based on how well the agent has achieved the target. However, the reward expectations in RegLinRS are linearly approximated based on features, which limits its applicability when the relationship between features and reward expectations is non-linear. To handle more complex environments, we proposed Neural Risk-sensitive Satisficing (NeuralRS), which incorporates neural networks into RegLinRS, and demonstrated its utility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。