用语义令牌替代物品ID，让推荐模型更易扩展且性能提升
Farewell to Item IDs: Unlocking the Scaling Potential of Large Ranking Models via Semantic Tokens
- 用物品的语义特征生成令牌，取代传统物品ID
- 存储量减少33%，AUC提升0.85%，模型扩容时表现更优
- 已在真实搜索系统上线，显著提升用户活跃与查询变化率
近期大规模推荐与搜索系统研究虽取得显著进展，但多数系统依赖物品ID，将每个物品视为独立类别符号并映射为嵌入向量。随着物品频繁增减，这些嵌入难以训练和维护，导致神经网络参数学习效率下降，限制了模型可扩展性。本文证明，相比物品ID，语义令牌具有更强的可扩展潜力。提出的TRM框架优化了令牌生成与应用流程，在保持性能的同时实现33%的稀疏存储减少，并带来0.85%的AUC提升。大量实验表明，当模型规模扩大时，TRM持续优于现有最优模型。该方法已在大规模个性化搜索系统中成功部署，通过A/B测试验证，用户活跃天数提升0.26%，查询变化率提高0.75%。
原文摘要 · Abstract (English)
Recent studies on scaling up ranking models have achieved substantial improvement for recommendation systems and search engines. However, most large-scale ranking systems rely on item IDs, where each item is treated as an independent categorical symbol and mapped to a learned embedding. As items rapidly appear and disappear, these embeddings become difficult to train and maintain. This instability impedes effective learning of neural network parameters and limits the scalability of ranking models. In this paper, we show that semantic tokens possess greater scaling potential compared to item IDs. Our proposed framework TRM improves the token generation and application pipeline, leading to 33% reduction in sparse storage while achieving 0.85% AUC increase. Extensive experiments further show that TRM could consistently outperform state-of-the-art models when model capacity scales. Finally, TRM has been successfully deployed on large-scale personalized search engines, yielding 0.26% and 0.75% improvement on user active days and change query ratio respectively through A/B test.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。