arXiv:2602.07526cs.IR2026-02KDD被引 1

用可扩展内存动态激活参数,让推荐模型更精准且快。

MSN: A Memory-based Sparse Activation Scaling Framework for Large-scale Industrial Recommendation

论文配图:MSN: A Memory-based Sparse Activation Scaling Framework for Large-scale Industrial Recommendation
图 1 · 摘自论文原文
  • 用记忆模块按需调取个性化特征,降低计算开销。
  • 通过分块键索引将检索复杂度从线性降到亚线性。
  • 已在抖音搜索系统落地,线上效果显著优于现有模型。

扩大深度学习推荐模型规模是提升表达能力的有效手段,但现有方法常带来巨大计算开销,难以在严苛延迟要求的工业系统中部署。现有稀疏激活方法如稀疏专家混合(Sparse Mixture-of-Experts)虽通过仅激活部分参数减少计算,但仍面临高内存访问成本和个性化能力受限的问题,因专家数量少、规模大。为此,我们提出基于内存的稀疏激活扩展框架 MSN。MSN 动态从大规模参数化记忆中检索个性化表示,并通过记忆门控机制融入下游特征交互模块,实现细粒度个性化的同时保持低计算开销。为在控制计算与内存访问成本的前提下进一步扩展内存容量,MSN 采用产品键记忆(Product-Key Memory, PKM)机制,将记忆检索复杂度从线性降至亚线性。此外,引入归一化与过参数化技术以维持内存利用率均衡,防止记忆检索崩溃。我们还设计了定制化的 Sparse-Gather 操作符,并采用 AirTopK 操作符,提升工业场景下的训练与推理效率。大量实验表明,MSN 在保持高效率的同时持续提升推荐性能。更重要的是,该框架已成功部署于抖音搜索排序系统,在离线指标和大规模在线 A/B 测试中均显著优于现有最优模型。

原文摘要 · Abstract (English)

Scaling deep learning recommendation models is an effective way to improve model expressiveness. Existing approaches often incur substantial computational overhead, making them difficult to deploy in large-scale industrial systems under strict latency constraints. Recent sparse activation scaling methods, such as Sparse Mixture-of-Experts, reduce computation by activating only a subset of parameters, but still suffer from high memory access costs and limited personalization capacity due to the large size and small number of experts. To address these challenges, we propose MSN, a memory-based sparse activation scaling framework for recommendation models. MSN dynamically retrieves personalized representations from a large parameterized memory and integrates them into downstream feature interaction modules via a memory gating mechanism, enabling fine-grained personalization with low computational overhead. To enable further expansion of the memory capacity while keeping both computational and memory access costs under control, MSN adopts a Product-Key Memory (PKM) mechanism, which factorizes the memory retrieval complexity from linear time to sub-linear complexity. In addition, normalization and over-parameterization techniques are introduced to maintain balanced memory utilization and prevent memory retrieval collapse. We further design customized Sparse-Gather operator and adopt the AirTopK operator to improve training and inference efficiency in industrial settings. Extensive experiments demonstrate that MSN consistently improves recommendation performance while maintaining high efficiency. Moreover, MSN has been successfully deployed in the Douyin Search Ranking System, achieving significant gains over deployed state-of-the-art models in both offline evaluation metrics and large-scale online A/B test.

推荐系统稀疏激活内存机制工业部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。