提出MARM模型，用记忆增强突破推荐系统算力瓶颈。
MARM: Unlocking the Future of Recommendation Systems through Memory Augmentation and Scalable Complexity
- 引入记忆增强机制，缓解高复杂度下的计算压力。
- 在日均500亿用户样本下实现超2000亿参数模型训练。
- 适合大规模推荐系统研发与算力优化研究者参考。
近年来，缩放定律指导了自然语言处理模型的设计，但其无法直接适用于推荐系统（RecSys），原因有二：其一，训练样本数量和模型参数量通常不是瓶颈；当前推荐系统每日可生成超过500亿用户样本，足以支持超过2000亿参数的模型，远超多数大语言模型（约1000亿参数）。其二，为保障推荐系统的稳定性与鲁棒性，必须严格控制计算复杂度（FLOPs）。基于上述差异，我们得出结论：对于推荐系统模型而言，相比模型参数量，计算复杂度FLOPs是更需谨慎控制的昂贵因素。本文提出里程碑工作MARM（Memory Augmented Recommendation Model），成功探索了一种新的缓存缩放定律。
原文摘要 · Abstract (English)
Scaling-law has guided the language model designing for past years, however, it is worth noting that the scaling laws of NLP cannot be directly applied to RecSys due to the following reasons: (1) The amount of training samples and model parameters is typically not the bottleneck for the model. Our recommendation system can generate over 50 billion user samples daily, and such a massive amount of training data can easily allow our model parameters to exceed 200 billion, surpassing many LLMs (about 100B). (2) To ensure the stability and robustness of the recommendation system, it is essential to control computational complexity FLOPs carefully. Considering the above differences with LLM, we can draw a conclusion that: for a RecSys model, compared to model parameters, the computational complexity FLOPs is a more expensive factor that requires careful control. In this paper, we propose our milestone work, MARM (Memory Augmented Recommendation Model), which explores a new cache scaling-laws successfully.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。