让推荐系统生成个性化解释，同时不降低推荐精度。
Shielded RecRL: Explanation Generation for Recommender Systems without Ranking Degradation
- 用双塔结构隔离语言模型与推荐模型，只微调少量参数。
- 解释长度、相关性与连贯性共同构成奖励，提升解释质量。
- 适合想增强推荐可解释性又怕影响排名的工程师。
我们提出Shielded RecRL，一种强化学习方法，可在不牺牲推荐系统原始排序性能的前提下生成个性化解释。不同于以往直接优化物品排序的基于RLHF的方法，我们的双塔架构保持推荐排序模型不变，仅让语言模型学习生成有帮助的解释。设计了融合解释长度、内容相关性和连贯性的复合奖励信号，并采用带KL散度约束的近端策略优化（PPO）算法，仅通过LoRA适配器微调大语言模型0.4%的参数。在包含约5万条交互数据的Amazon Books幻想与浪漫类图书数据集上，Shielded RecRL将相对点击率（CTR）提升22.5%（达基线的1.225倍），同时几乎完全保留推荐系统的原始排序行为。大量消融实验证实，梯度屏蔽策略和奖励设计能有效平衡解释质量与策略漂移。结果表明，Shielded RecRL通过丰富且个性化的解释提升了推荐的用户感知价值，而未损害核心推荐准确性。
原文摘要 · Abstract (English)
We introduce Shielded RecRL, a reinforcement learning approach to generate personalized explanations for recommender systems without sacrificing the system's original ranking performance. Unlike prior RLHF-based recommender methods that directly optimize item rankings, our two-tower architecture keeps the recommender's ranking model intact while a language model learns to produce helpful explanations. We design a composite reward signal combining explanation length, content relevance, and coherence, and apply proximal policy optimization (PPO) with a KL-divergence constraint to fine-tune a large language model with only 0.4% of its parameters trainable via LoRA adapters. In experiments on an Amazon Books dataset (approximately 50K interactions in the fantasy and romance genres), Shielded RecRL improved the relative click-through rate (CTR) by 22.5% (1.225x over baseline) while keeping the recommender's item-ranking behavior virtually unchanged. An extensive ablation study confirms that our gradient shielding strategy and reward design effectively balance explanation quality and policy drift. Our results demonstrate that Shielded RecRL enhances user-facing aspects of recommendations through rich, personalized explanations without degrading core recommendation accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。