arXiv:2505.16810cs.IR2025-05被引 4

让大模型与推荐系统对话多轮,深度挖掘物品空间。

DeepRec: Towards a Deep Dive Into the Item Space with Large Language Model Based Recommendation

  • 大模型与传统推荐系统多轮对话,自主探索物品
  • 多轮交互后排序候选物品,显著提升推荐效果
  • 适合想深入理解推荐机制的科研与工程人员

近期,大语言模型(LLMs)被引入推荐系统(RSs),或用于增强传统推荐模型(TRMs),或作为推荐主干。然而,现有基于大模型的推荐系统未能充分融合大模型(如世界知识和推理能力)与传统推荐模型(如推荐专用知识和效率)的互补优势,以深入探索物品空间。为此,我们提出DeepRec,一种新型基于大模型的推荐系统,通过大模型与传统推荐模型之间的自主多轮交互,实现对物品空间的深度探索。在每轮交互中,大模型基于用户偏好进行推理,并与传统推荐模型协同检索候选物品。经过多轮交互后,大模型对检索到的物品进行排序,生成最终推荐结果。我们采用强化学习(RL)优化策略,并从三个维度提出新设计:基于推荐模型的数据轨迹生成、面向推荐任务的分层奖励机制,以及两阶段强化学习训练策略。在数据轨迹生成方面,引入感知偏好的传统推荐模型,使大模型可与其交互构建轨迹数据;在奖励设计方面,构建包含过程级与结果级奖励的分层奖励函数,分别优化交互过程与推荐性能;在训练策略上,采用两阶段训练:第一阶段引导大模型与传统推荐模型有效互动,第二阶段聚焦性能提升。在公开数据集上的实验表明,DeepRec显著优于传统及基于大模型的基线方法,为推荐系统中的深度探索提供了新范式。

原文摘要 · Abstract (English)

Recently, large language models (LLMs) have been introduced into recommender systems (RSs), either to enhance traditional recommendation models (TRMs) or serve as recommendation backbones. However, existing LLM-based RSs often do not fully exploit the complementary advantages of LLMs (e.g., world knowledge and reasoning) and TRMs (e.g., recommendation-specific knowledge and efficiency) to fully explore the item space. To address this, we propose DeepRec, a novel LLM-based RS that enables autonomous multi-turn interactions between LLMs and TRMs for deep exploration of the item space. In each interaction turn, LLMs reason over user preferences and interact with TRMs to retrieve candidate items. After multi-turn interactions, LLMs rank the retrieved items to generate the final recommendations. We adopt reinforcement learning(RL) based optimization and propose novel designs from three aspects: recommendation model based data rollout, recommendation-oriented hierarchical rewards, and a two-stage RL training strategy. For data rollout, we introduce a preference-aware TRM, with which LLMs interact to construct trajectory data. For rewards, we design a hierarchical reward function that involves both process-level and outcome-level rewards to optimize the interaction process and recommendation performance, respectively. For RL training, we develop a two-stage training strategy, where the first stage aims to guide LLMs to interact with TRMs and the second stage focuses on performance improvement. Experiments on public datasets demonstrate that DeepRec significantly outperforms both traditional and LLM-based baselines, offering a new paradigm for deep exploration in recommendation systems.

推荐系统大模型强化学习多轮交互

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。