arXiv:2604.10029cs.IR2026-04被引 1

让推荐智能体与用户智能体共同进化,通过自蒸馏强化学习提升推荐精准度。

Self-Distilled Reinforcement Learning for Co-Evolving Agentic Recommender Systems

论文配图:Self-Distilled Reinforcement Learning for Co-Evolving Agentic Recommender Systems
图 1 · 摘自论文原文
  • 设计双智能体交互奖励机制,利用对话轨迹生成联合监督信号。
  • 提出自蒸馏信用分配方法,将历史对话转化为细粒度学习信号。
  • 在多个数据集上显著优于现有推荐系统,适合构建高互动性推荐应用。

大语言模型驱动的智能体推荐系统(ARS)将推荐过程重构为推荐智能体与用户智能体之间的多轮交互,实现超越传统单次预测的迭代偏好挖掘与优化。然而,现有ARS主要采用反思式范式,依赖外部文本记忆存储过往交互轨迹,导致经验无法内化至模型参数,智能体仍依赖通用推理而非持续学习特定推荐能力。强化学习(RL)可将交互经验参数化,但现有方法存在两大局限:一是忽略智能体间动态互扰的交互本质,难以生成内生监督;二是将丰富的多轮交互压缩为最终结果,丢失轨迹中的密集监督信息。为此,本文提出CoARS——一种用于共演化智能体推荐系统的自蒸馏强化学习框架。CoARS引入两种互补学习机制:交互奖励,从同一交互轨迹中提取推荐与用户智能体的任务级联合监督;自蒸馏信用分配,在教师-学生框架下将历史轨迹转化为词元级信用信号。多数据集实验表明,CoARS在推荐性能与用户对齐度上均优于代表性基准模型。

原文摘要 · Abstract (English)

Large language model-empowered agentic recommender systems (ARS) reformulate recommendation as a multi-turn interaction between a recommender agent and a user agent, enabling iterative preference elicitation and refinement beyond conventional one-shot prediction. However, existing ARS are mainly optimized in a Reflexion-style paradigm, where past interaction trajectories are stored as textual memory and retrieved as prompt context for later reasoning. Although this design allows agents to recall prior feedback and observations, the accumulated experience remains external to model parameters, leaving agents reliant on generic reasoning rather than progressively acquiring recommendation-specific decision-making ability through learning. Reinforcement learning (RL) therefore provides a natural way to internalize such interaction experience into parameters. Yet existing RL methods for ARS still suffer from two key limitations. First, they fail to capture the interactive nature of ARS, in which the recommender agent and the user agent continuously influence each other and can naturally generate endogenous supervision through interaction feedback. Second, they reduce a rich multi-turn interaction process to final outcomes, overlooking the dense supervision embedded throughout the trajectory. To this end, we propose CoARS, a self-distilled reinforcement learning framework for co-evolving agentic recommender systems. CoARS introduces two complementary learning schemes: interaction reward, which derives coupled task-level supervision for the recommender agent and the user agent from the same interaction trajectory, and self-distilled credit assignment, which converts historical trajectories into token-level credit signals under teacher-student conditioning. Experiments on multiple datasets show that CoARS outperforms representative ARS baselines in recommendation performance and user alignment.

推荐系统强化学习智能体自蒸馏

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。