针对生成式推荐高并发低延迟需求，提出高效服务系统xGR
xGR: Efficient Generative Recommendation Serving at Scale
- 通过分阶段计算与分离缓存统一预填充和解码流程
- 实现早期排序终止与掩码过滤，降低长物品空间排序开销
- 重构流水线支持多级重叠与多流并行，适合大规模推荐场景
推荐系统通过个性化预测带来显著经济价值。生成式推荐（GR）融合大语言模型以增强对长用户-物品序列的理解。尽管采用基于注意力的架构，但GR的工作负载与大语言模型服务差异显著：通常处理长提示词，生成短而固定的输出，且因较大束宽导致每步解码计算成本极高。此外，由于束搜索涉及庞大物品空间，排序开销尤为耗时。本文提出xGR，一种面向生成式推荐的服务系统，在高并发场景下满足严格低延迟要求。首先，xGR通过分阶段计算与分离键值缓存统一预填充与解码阶段处理；其次，通过数据结构复用实现早期排序终止与基于掩码的物品过滤；第三，重构整体流水线以利用多级重叠与多流并行。在真实数据集上的实验表明，在严格延迟约束下，xGR相比最先进基线系统吞吐量提升至少2.89倍。
原文摘要 · Abstract (English)
Recommendation system delivers substantial economic benefits by providing personalized predictions. Generative recommendation (GR) integrates LLMs to enhance the understanding of long user-item sequences. Despite employing attention-based architectures, GR's workload differs markedly from that of LLM serving. GR typically processes long prompt while producing short, fixed-length outputs, yet the computational cost of each decode phase is especially high due to the large beam width. Furthermore, since the beam search involves a vast item space, the sorting overhead becomes particularly time-consuming. We propose xGR, a GR-oriented serving system that meets strict low-latency requirements under high-concurrency scenarios. First, xGR unifies the processing of prefill and decode phases through staged computation and separated KV cache. Second, xGR enables early sorting termination and mask-based item filtering with data structure reuse. Third, xGR reconstructs the overall pipeline to exploit multi-level overlap and multi-stream parallelism. The experiments on real-world datasets demonstrate that xGR achieves at least 2.89x throughput compared to the state-of-the-art baseline under strict latency constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。