提出UG-Sep框架,让推荐模型推理时用户计算只算一次,大幅降延迟。
Compute Only Once: UG-Separation for Efficient Large Recommendation Models
- 将用户和物品特征在混合层显式分离,保留纯用户表征供复用
- 线上实测推理延迟降低20%,且不影响用户体验和商业指标
- 适合大规模推荐系统,尤其适用于高并发场景下的高效部署
随着推荐系统规模扩大,模型复杂度上升导致训练与推理成本剧增。尽管长序列模型可通过键值缓存复用用户侧计算,但在基于TokenMixer的密集交互架构中,用户与物品特征深度耦合,难以实现计算复用。本文提出用户-群体分离(UG-Sep)框架,首次在TokenMixer结构中实现用户侧计算可复用。该方法在混合层显式分离用户与物品信息流,确保部分令牌在多层间保持纯用户表征,使其计算可跨样本复用,显著减少冗余开销。为弥补因掩码带来的表达能力损失,进一步提出信息补偿策略,自适应重建被抑制的用户-物品交互。此外,由于UG-Sep大幅降低用户侧浮点运算量并暴露内存瓶颈,引入W8A16(8-bit权重,16-bit激活)权重量化以缓解带宽压力,实现额外加速。我们在字节跳动进行大规模离线评估与在线A/B实验,涵盖抖音信息流推荐、火山信息流推荐、穿山甲广告与千川广告等多个核心业务场景。结果表明,相比原有TokenMixer模型,UG-Sep在不改变线上用户体验及商业指标的前提下,推理延迟最高降低20%。
原文摘要 · Abstract (English)
Driven by scaling laws, recommender systems increasingly rely on larger-scale models to capture complex feature interactions and user behaviors, but this trend also leads to prohibitive training and inference costs. While long-sequence models can reuse user-side computation through KV Caching, such reuse is difficult in TokenMixer-based dense feature interaction architectures, where user and group features are deeply entangled and mixed-up across layers. In this work, we present User-Group Separation (UG-Sep), an industrial large-scale framework that enables user-side computation reusable in TokenMixer-based dense interaction models for the first time. UG-Sep explicitly disentangles user-side and item-side information flows within token-mixing layers, ensuring that a subset of tokens preserves purely user-side representations across layers. This design allows the corresponding per-token computations to be reused across multiple samples, significantly reducing redundant inference cost. To compensate for the potential expressive capacity loss induced by masking, we further propose an Information Compensation strategy that adaptively reconstructs suppressed user-item interactions. Moreover, as UG-Sep substantially reduces user-side FLOPs and exposes memory-bound components, we incorporate W8A16 (8-bit weight, 16-bit activation) weight-only quantization to alleviate memory bandwidth bottlenecks and achieve additional acceleration. We conduct extensive offline evaluations and large-scale online A/B experiments at ByteDance to validate the effectiveness of UG-Sep. Results show that UG-Sep reduces inference latency by up to 20% without causing adverse changes to online user experience and commercial metrics on multiple influential business scenarios compared to TokenMixer at ByteDance, including Douyin Feed Recommendation, Hongguo Feed Recommendation, Chuanshanjia Ads, and Qianchuan Ads.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。