通过稀疏专家路由提升推荐系统可扩展性,解决多任务模型扩容瓶颈。
SMES: Towards Scalable Multi-Task Recommendation via Expert Sparsity
- 设计分层稀疏专家结构,共享专家与私有专家协同控制计算量。
- 线上实验实现GAUC提升0.29%,用户观看时长增加0.31%。
- 适合大规模工业推荐系统,尤其适用于稀疏任务场景。
工业级推荐系统通常依赖多任务学习来建模多样化的用户反馈信号并聚合用于排序。尽管模型规模扩大带来了显著收益,但盲目增加参数量会带来高昂的在线推理成本,且对标签分布偏斜的稀疏任务效果递减。这种统一参数扩展与异构任务需求之间的矛盾,构成了可扩展多任务推荐的根本挑战。本文研究参数稀疏化作为系统性扩展范式,识别出将稀疏混合专家(MoE)应用于多任务推荐时的两大障碍:爆炸式专家激活破坏实例级稀疏性,以及独立任务路由导致的专家负载不均。为此提出SMES框架,采用渐进式专家路由机制,将专家激活分解为跨任务共享的专家子集与任务自适应的私有专家,显式控制单实例执行专家数量,同时保留任务特异性容量。此外引入全局多门控负载均衡正则项,稳定训练过程,调节所有任务下专家总体利用率。SMES已在快手大规模短视频服务中部署,支持超4亿日活用户。大量线上实验表明系统性提升,GAUC提升0.29%,用户观看时长增加0.31%。
原文摘要 · Abstract (English)
Industrial recommender systems typically rely on multi-task learning to estimate diverse user feedback signals and aggregate them for ranking. Recent advances in model scaling have shown promising gains in recommendation. However, naively increasing model capacity imposes prohibitive online inference costs and often yields diminishing returns for sparse tasks with skewed label distributions. This mismatch between uniform parameter scaling and heterogeneous task capacity demands poses a fundamental challenge for scalable multi-task recommendation. In this work, we investigate parameter sparsification as a principled scaling paradigm and identify two critical obstacles when applying sparse Mixture-of-Experts (MoE) to multi-task recommendation: exploded expert activation that undermines instance-level sparsity and expert load skew caused by independent task-wise routing. To address these challenges, we propose SMES, a scalable sparse MoE framework with progressive expert routing. SMES decomposes expert activation into a task-shared expert subset jointly selected across tasks and task-adaptive private experts, explicitly bounding per-instance expert execution while preserving task-specific capacity. In addition, SMES introduces a global multi-gate load-balancing regularizer that stabilizes training by regulating aggregated expert utilization across all tasks. SMES has been deployed in Kuaishou large-scale short-video services, supporting over 400 million daily active users. Extensive online experiments demonstrate stable improvements, with GAUC gain of 0.29% and a 0.31% uplift in user watch time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。