美团提出MTGR框架,让生成式推荐模型保留交叉特征并高效扩展。
MTGR: Industrial-Scale Generative Recommendation Framework in Meituan
- 基于HSTU架构保留深度推荐模型的交叉特征
- 单样本前向计算达DLRM的65倍加速,训练推理更高效
- 适合大规模推荐系统部署,尤其适合高并发场景
缩放定律在自然语言处理和计算机视觉等领域已得到广泛验证。在推荐系统中,近期工作采用生成式推荐以实现可扩展性,但其方法需放弃传统推荐模型精心构建的交叉特征,导致性能显著下降,且无法通过扩大规模弥补。本文提出MTGR(Meituan Generative Recommendation)框架,基于HSTU架构,可保留原始深度学习推荐模型(DLRM)的特征,包括交叉特征。通过用户级压缩实现训练与推理加速,并提出分组层归一化(GLN)提升不同语义空间内的编码性能,以及动态掩码策略防止信息泄露。进一步优化训练框架,使模型支持10至100倍于DLRM的计算复杂度,而成本增加有限。MTGR在单样本前向计算中达到DLRM的65倍FLOPs,近两年内线上与离线均取得最大提升。该成果已成功部署于全球最大的外卖平台美团,支撑主要流量。
原文摘要 · Abstract (English)
Scaling law has been extensively validated in many domains such as natural language processing and computer vision. In the recommendation system, recent work has adopted generative recommendations to achieve scalability, but their generative approaches require abandoning the carefully constructed cross features of traditional recommendation models. We found that this approach significantly degrades model performance, and scaling up cannot compensate for it at all. In this paper, we propose MTGR (Meituan Generative Recommendation) to address this issue. MTGR is modeling based on the HSTU architecture and can retain the original deep learning recommendation model (DLRM) features, including cross features. Additionally, MTGR achieves training and inference acceleration through user-level compression to ensure efficient scaling. We also propose Group-Layer Normalization (GLN) to enhance the performance of encoding within different semantic spaces and the dynamic masking strategy to avoid information leakage. We further optimize the training frameworks, enabling support for our models with 10 to 100 times computational complexity compared to the DLRM, without significant cost increases. MTGR achieved 65x FLOPs for single-sample forward inference compared to the DLRM model, resulting in the largest gain in nearly two years both offline and online. This breakthrough was successfully deployed on Meituan, the world's largest food delivery platform, where it has been handling the main traffic.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。