GenRec通过生成式框架提升推荐系统效果,解决大规模场景下的输出不一致与偏好对齐难题。
GenRec: A Preference-Oriented Generative Framework for Large-Scale Recommendation
- 采用分页级下一词预测,统一处理整页交互,减少输出波动。
- 输入压缩使序列长度减半,精度损失可忽略,加速推理效率。
- 结合强化学习与混合奖励机制,更精准匹配用户真实偏好,适合工业级推荐系统。
生成式召回(GR)通过下一词预测(NTP)为推荐提供新范式。但在大规模工业系统中面临三大挑战:(i)单次请求因分页机制导致相同输入产生不一致输出;(ii)基于语义ID的多标记项表示在长用户行为序列编码上成本过高;(iii)生成策略与细微用户偏好信号难以对齐。本文提出GenRec,一个部署于京东App的面向偏好的生成式框架,集成于单一解码器架构中。训练目标上,提出分页级NTP任务,以整页交互为监督单位,提供更密集梯度信号,消除点级训练的一对多模糊性。预填充阶段,引入非对称线性令牌合并(Token Merger),压缩提示中的多标记语义ID,同时保持全分辨率解码,输入长度减少约2倍,精度损失微乎其微。为更好对齐用户满意度,提出GRPO-SR,结合组相对策略优化与负对数似然正则化,提升训练稳定性,并采用混合奖励机制,融合密集奖励模型与相关性门控,缓解奖励欺骗问题。在持续一个月的线上A/B测试中,GenRec相比现有流水线,点击量提升9.5%,交易量提升8.7%。
原文摘要 · Abstract (English)
Generative Retrieval (GR) offers a promising paradigm for recommendation through next-token prediction (NTP). However, scaling it to large-scale industrial systems introduces three challenges: (i) within a single request, the identical model inputs may produce inconsistent outputs due to the pagination request mechanism; (ii) the prohibitive cost of encoding long user behavior sequences with multi-token item representations based on semantic IDs, and (iii) aligning the generative policy with nuanced user preference signals. We present GenRec, a preference-oriented generative framework deployed on the JD App that addresses above challenges within a single decoder-only architecture. For training objective, we propose Page-wise NTP task, which supervises over an entire interaction page rather than each interacted item individually, providing denser gradient signal and resolving the one-to-many ambiguity of point-wise training. On the prefilling side, an asymmetric linear Token Merger compresses multi-token Semantic IDs in the prompt while preserving full-resolution decoding, reducing input length by ~2X with negligible accuracy loss. To further align outputs with user satisfaction, we introduce GRPO-SR, a reinforcement learning method that pairs Group Relative Policy Optimization with NLL regularization for training stability, and employs Hybrid Rewards combining a dense reward model with a relevance gate to mitigate reward hacking. In month-long online A/B tests serving production traffic, GenRec achieves 9.5% improvement in click count and 8.7% in transaction count over the existing pipeline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。