用单阶段生成器取代传统两阶段排序,提升推荐效果。
GoalRank: Group-Relative Optimization for a Large Ranking Model
- 仅用生成器的单阶段框架,避免评价器瓶颈。
- 在多个公开数据集上优于现有方法,线上测试表现更优。
- 适合大规模推荐系统,尤其适用于追求高效部署的场景。
主流排序方法多采用生成-评估两阶段范式:生成器产出候选列表,评估器选出最优。近期研究尝试通过多生成器扩大候选集以提升性能,但排序空间呈组合爆炸,单纯增加候选数量效果有限且很快饱和。与此同时,大推荐模型的发展表明,端到端单阶段模型可借助缩放定律实现良好表现。受此启发,我们从仅生成器的单阶段视角重新审视排序问题。理论上证明:对任意(有限)生成-评估模型,总存在一个仅生成器模型,其逼近最优排序策略的误差严格更小,且随规模增大仍具缩放性。基于此,我们推导出单阶段优化目标的证据上界,发现可利用真实用户反馈训练的奖励模型,以群体相对方式构建参考策略。该策略作为最优策略的实用替代,有效指导大规模生成器排序器的训练。据此提出GoalRank框架。大量离线实验与大规模线上A/B测试均显示,GoalRank持续超越现有先进方法。
原文摘要 · Abstract (English)
Mainstream ranking approaches typically follow a Generator-Evaluator two-stage paradigm, where a generator produces candidate lists and an evaluator selects the best one. Recent work has attempted to enhance performance by expanding the number of candidate lists, for example, through multi-generator settings. However, ranking involves selecting a recommendation list from a combinatorially large space. Simply enlarging the candidate set remains ineffective, and performance gains quickly saturate. At the same time, recent advances in large recommendation models have shown that end-to-end one-stage models can achieve promising performance with the expectation of scaling laws. Motivated by this, we revisit ranking from a generator-only one-stage perspective. We theoretically prove that, for any (finite Multi-)Generator-Evaluator model, there always exists a generator-only model that achieves strictly smaller approximation error to the optimal ranking policy, while also enjoying scaling laws as its size increases. Building on this result, we derive an evidence upper bound of the one-stage optimization objective, from which we find that one can leverage a reward model trained on real user feedback to construct a reference policy in a group-relative manner. This reference policy serves as a practical surrogate of the optimal policy, enabling effective training of a large generator-only ranker. Based on these insights, we propose GoalRank, a generator-only ranking framework. Extensive offline experiments on public benchmarks and large-scale online A/B tests demonstrate that GoalRank consistently outperforms state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。