用强化学习让大模型直接学推荐，不靠昂贵数据还能保持通用能力。
Rec-R1: Bridging Generative Large Language Models and User-Centric Recommendation Systems via Reinforcement Learning
- 通过闭环反馈优化大模型生成，不依赖合成训练数据。
- 在商品搜索和序列推荐上超越提示和微调方法，甚至胜过强基线。
- 保留大模型的通用能力，适合持续个性化适应场景。
我们提出 Rec-R1，一种通用的强化学习框架，通过闭环优化将大语言模型（LLMs）与推荐系统连接。不同于提示工程和监督微调（SFT），Rec-R1利用固定黑箱推荐模型的反馈直接优化 LLM 生成，无需依赖 GPT-4o 等专有模型生成的合成 SFT 数据，从而避免高昂的数据蒸馏成本与努力。为验证 Rec-R1 的有效性，我们在两个代表性任务上进行评估：商品搜索与序列推荐。实验结果表明，Rec-R1 不仅持续优于提示与 SFT 方法，还在使用简单检索器（如 BM25）时显著超越强判别基线。此外，与 SFT 会损害指令遵循与推理能力不同，Rec-R1 保持了 LLM 的通用性。这些发现表明，Rec-R1 是实现无灾难性遗忘的持续任务适配的有力基础。
原文摘要 · Abstract (English)
We propose Rec-R1, a general reinforcement learning framework that bridges large language models (LLMs) with recommendation systems through closed-loop optimization. Unlike prompting and supervised fine-tuning (SFT), Rec-R1 directly optimizes LLM generation using feedback from a fixed black-box recommendation model, without relying on synthetic SFT data from proprietary models such as GPT-4o. This avoids the substantial cost and effort required for data distillation. To verify the effectiveness of Rec-R1, we evaluate it on two representative tasks: product search and sequential recommendation. Experimental results demonstrate that Rec-R1 not only consistently outperforms prompting- and SFT-based methods, but also achieves significant gains over strong discriminative baselines, even when used with simple retrievers such as BM25. Moreover, Rec-R1 preserves the general-purpose capabilities of the LLM, unlike SFT, which often impairs instruction-following and reasoning. These findings suggest Rec-R1 as a promising foundation for continual task-specific adaptation without catastrophic forgetting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。