用大模型指导推荐系统，提升新颖性和多样性。
Large Language Model-Enhanced Reinforcement Learning for Diverse and Novel Recommendations
- 用大模型生成新奇推荐，小模型基于数据优化建议。
- 在真实数据上显著提升推荐多样性与新颖性，准确率不降。
- 无需微调大模型，适合数据不平衡场景使用。
在推荐系统中，多样性与新颖性对捕捉用户偏好、促进探索至关重要，但许多系统仍侧重点击相关性。虽然强化学习已被用于提升多样性，但常依赖随机探索，可能偏离用户兴趣。本文提出LAAC（LLM引导的对抗演员-评论家）方法，利用大语言模型作为参考策略生成新颖项目建议，并训练轻量级策略模型结合系统特定数据优化这些建议。该方法将训练建模为演员与评论家网络间的双层优化，使评论家能选择性地青睐有潜力的新颖动作，演员则超越大模型建议持续改进策略。为缓解不可靠大模型建议带来的高估问题，引入正则化，将未探索项目的评论家值锚定在已知良好动作附近。在真实数据集上的实验表明，LAAC在多样性、新颖性和准确性上均优于现有基线，且对不平衡数据具有鲁棒性，有效融合大模型知识而无需昂贵微调。
原文摘要 · Abstract (English)
In recommendation systems, diversity and novelty are essential for capturing varied user preferences and encouraging exploration, yet many systems prioritize click relevance. While reinforcement learning (RL) has been explored to improve diversity, it often depends on random exploration that may not align with user interests. We propose LAAC (LLM-guided Adversarial Actor Critic), a novel method that leverages large language models (LLMs) as reference policies to suggest novel items, while training a lightweight policy to refine these suggestions using system-specific data. The method formulates training as a bilevel optimization between actor and critic networks, enabling the critic to selectively favor promising novel actions and the actor to improve its policy beyond LLM recommendations. To mitigate overestimation of unreliable LLM suggestions, we apply regularization that anchors critic values for unexplored items close to well-estimated dataset actions. Experiments on real-world datasets show that LAAC outperforms existing baselines in diversity, novelty, and accuracy, while remaining robust on imbalanced data, effectively integrating LLM knowledge without expensive fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。