arXiv:2607.25344cs.IR2026-07

让推荐生成更懂业务价值,动态调整推荐结果排序

Reward Guided Decoding for Generative Recommendation

论文配图:Reward Guided Decoding for Generative Recommendation
图 1 · 摘自论文原文
  • 引入奖励模型引导解码,在不重训练模型前提下调整推荐路径
  • 在线实验显示在快手平台显著提升业务价值指标
  • 适合需要快速响应业务策略变化的工业级推荐场景

生成式推荐将推荐任务建模为序列自回归生成,但解码过程常被生成概率主导,导致高价值候选品因生成概率低而被提前剪枝。现有重排序或训练阶段对齐方法要么干预过晚,要么在业务偏好变化时需昂贵的模型重训练。为此,我们提出奖励引导解码(RGD),一种面向工业价值的可控解码框架。将价值导向解码建模为KL正则化的奖励最大化问题,推导出闭式奖励引导解码分布,从原理上融合生成概率与奖励信号。RGD将基础生成器视为参考策略,引入奖励模型作为运行时控制器,在每一步解码中注入奖励信号,重塑搜索轨迹而无需重训练生成器。大量离线与在线实验验证了该方法在个性化与商业价值对齐上的有效性。RGD已在快手平台部署,持续提升真实推荐场景表现。

原文摘要 · Abstract (English)

Generative recommendation formulates recommendation task into an SID sequence autoregressive generation paradigm, but the decoding process is often dominated by generation likelihood. This may conflict with real-world business objectives, where high-value candidates can receive low generation probability and be pruned early during beam search. Existing reranking or training-time alignment methods either intervene too late or require costly model retraining when business preferences change. To this end, we propose \textbf{R}eward \textbf{G}uided \textbf{D}ecoding, named \textbf{RGD}, a controllable decoding framework for industrial value-oriented generative recommendation. We formulate value-guided decoding as a KL-regularized reward maximization problem, deriving a closed-form reward guided decoding distribution that principledly combines generation probability with reward signals. RGD treats the base generator as a reference policy and introduces a reward model as a test-time controller, injecting reward into each decoding step to reshape the search trajectory without retraining the generator. Extensive offline and online experiments demonstrate the effectiveness of our approach for aligning personalization and business value. RGD has been deployed on the Kuaishou platform, bringing consistent improvements in real-world recommendation scenarios.

生成推荐奖励引导工业部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。