用用户行为反馈训练模型生成推荐友好的描述,无需用户输入也能提升推荐效果。
RecoReward: Recommender-Guided Multimodal Description Generation for Recommendation

- 通过行为数据生成奖励信号,指导模型学习推荐相关描述。
- 在7个召回指标上超越基线模型,线上测试也表现更好。
- 适合需要高效生成通用推荐特征的场景,如直播推荐系统。
多模态大语言模型(MLLM)可将多模态商品内容转化为结构化描述,作为推荐系统的语义特征。传统仅基于内容的生成方法无法利用下游用户信号来确定应强调哪些语义。近期用户条件化方法虽引入用户历史或画像,但需在推理时提供用户信息,导致生成结果依赖用户。本文提出RecoReward,采用训练阶段的行为衍生奖励,保持内容驱动的推理方式。在直播推荐场景中,以历史参与用户作为未来目标用户的代理,利用非目标用户观测数据估算跨用户共有的亲和力。推荐亲和度得分(RAS)对比这些信号,为强化学习提供用户选择性反馈,使模型学习生成无需用户输入的共享描述。离线基准测试显示,RecoReward-9B在七个召回指标上均优于Qwen3.5-9B基线及其他评估模型;线上A/B测试也验证了性能提升。结果表明,RecoReward能训练出有利于下游推荐的物品特征,同时保持内容驱动的推理模式。
原文摘要 · Abstract (English)
Multimodal large language models (MLLMs) can convert multimodal item content into structured descriptions used as semantic features for recommendation. Conventional content-only generation, however, cannot use downstream user signals to determine which semantics should be emphasized. Recent user-conditioned methods incorporate these signals through user histories or profiles, but they require user information at inference and make generation user-dependent. In this paper, we introduce RecoReward, which instead uses behavior-derived rewards during training and preserves content-only inference. To instantiate this idea in live-stream recommendation, we treat historically engaged users as a proxy for future target users and use observational non-target users to estimate affinity shared broadly across users. The Recommender Affinity Score (RAS) contrasts these signals to provide user-selective feedback for reinforcement learning, allowing the learned policy to generate a single shared description without user inputs. In our offline benchmark, RecoReward-9B outperforms its Qwen3.5-9B baseline and all other evaluated models across seven recall metrics. Online A/B testing also shows performance gains. These results show that RecoReward trains the MLLM to produce item features that benefit downstream recommendation while retaining content-only serving.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。