用分层强化学习分离用户长期感知与短期兴趣，提升推荐效果。
Hierarchical Reinforcement Learning for Temporal Abstraction of Listwise Recommendation
- 分层结构：高层学用户长期感知，底层做短时物品选择决策。
- 实验显示性能显著优于多个基线方法。
- 适合研究长期用户行为建模的推荐系统开发者。
现代列表式推荐系统需兼顾用户的长期感知与短期兴趣变化。尽管强化学习可用于此类问题，但面临搜索空间大、用户反馈稀疏及交互延迟长等挑战。受分层强化学习进展启发，我们提出一种新框架mccHRL，为列表式推荐提供多层级的时间抽象能力。高层代理关注用户感知的演化，低层代理则将推荐过程建模为序列决策问题以生成物品选择策略。该框架实现了会话外上下文（由高层编码）与会话内上下文（由低层编码）的清晰分解。通过构建仿真环境和基于工业数据集的实验验证，结果表明该方法在性能上显著优于多个主流基线。相关数据与代码已公开。
原文摘要 · Abstract (English)
Modern listwise recommendation systems need to consider both long-term user perceptions and short-term interest shifts. Reinforcement learning can be applied on recommendation to study such a problem but is also subject to large search space, sparse user feedback and long interactive latency. Motivated by recent progress in hierarchical reinforcement learning, we propose a novel framework called mccHRL to provide different levels of temporal abstraction on listwise recommendation. Within the hierarchical framework, the high-level agent studies the evolution of user perception, while the low-level agent produces the item selection policy by modeling the process as a sequential decision-making problem. We argue that such framework has a well-defined decomposition of the outra-session context and the intra-session context, which are encoded by the high-level and low-level agents, respectively. To verify this argument, we implement both a simulator-based environment and an industrial dataset-based experiment. Results observe significant performance improvement by our method, compared with several well-known baselines. Data and codes have been made public.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。