用动态规划改进推荐系统,让推荐更连贯。
Planning over Matrix-Factorization MDPs for Candidate Generation

- 将推荐视为状态转移过程,用矩阵分解后验建模用户变化
- 单步前瞻就显著提升效果,尤其在电影和社交数据集上
- 无需重训练,适合想优化现有推荐系统的开发者
在推荐服务中,用户旅程可视为一系列物品推荐的链式过程:一个有用物品会改变用户状态,进而影响下一步应推荐的内容。标准矩阵分解检索忽略这一点——它构建单一用户向量,通过静态得分返回前K个物品,视其为独立。本文提出一个具体问题:何时值得对这种用户状态动态进行规划?为此,我们把前K项检索建模为基于隐式ALS后验$(A^{-1}, u)$的马尔可夫决策过程(MDP),其中动作是推荐物品,转移为闭式秩一折叠更新,轨迹奖励结合相关性相似度与后验对齐项。在五个数据集和两种协议下(每用户留最后n项、全局时间划分),对比静态检索、一步规划与视野为K的MCTS。在留最后n项设置下,动态感知规划在所有数据集上优于静态检索;在MovieLens-1M和VK-LSVD子集的全局时间划分下,增益依然显著。单步前瞻已捕获大部分收益,因此轻量级规划层可将静态前K评分升级为短决策流程,在不重训练、不改表示的前提下,提升基于固定协同过滤嵌入的检索性能。这些增益依赖于使用余弦相似度衡量相关性,而非内积——后者会与物品流行度混淆。
原文摘要 · Abstract (English)
For a recommender service, we view the customer journey as a chain of item recommendations: a useful item changes the user's state and therefore what should be retrieved next. Standard matrix-factorization retrieval ignores this -- it builds one user vector and returns the top-$K$ items by a static score, treating them as independent. We ask a narrow question: when is it worth planning over the user-state dynamics that fold-in induces? To answer it we propose casting top-$K$ retrieval as an MDP over the implicit-ALS posterior $(A^{-1},u)$, where an action is an item and the transition is a closed-form rank-one fold-in, and the trajectory reward combines a relevance similarity with a posterior-alignment term. Under the same fixed embeddings we compare static retrieval, one-step planning, and horizon-$K$ MCTS across five datasets and two protocols: a per-user leave-last-$n$ split and a stricter global time split. Dynamics-aware planning tends to overcome static retrieval on all datasets under leave-last-$n$, and the gains hold on MovieLens-1M and the VK-LSVD slices under the global time split. A single step of lookahead already captures most of the gain, so the lightweight planning layer turns static top-$K$ scoring into a short decision and improves retrieval over fixed collaborative-filtering embeddings, with no retraining and no change to the representation. These gains depend on measuring relevance with cosine rather than inner-product similarity, which is otherwise entangled with item popularity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。