用语义ID和压缩注意力模型,实现百亿用户超长视频推荐序列建模。
Beyond Item IDs: Scaling Short-Form-Video Recommendation via Semantic-Native Long Sequence Modeling

- 用内容语义生成的短ID替代传统视频ID,缩小嵌入表规模。
- 通过全局感知压缩变换器,使序列处理内存降低十倍、计算开销大幅减少。
- 适合大规模短视频推荐系统优化,尤其在冷启动和长序列场景下表现优异。
捕捉用户在海量观看历史中的兴趣对短视频推荐至关重要,但序列长度扩展受限于两个瓶颈:原子视频ID的语义稀疏性与Transformer的二次计算复杂度。传统正交视频ID无法捕捉内容关联,且需庞大的嵌入表;自注意力的二次复杂度在严格的工业延迟与资源约束下限制了最大序列长度。本文提出一个面向百亿用户规模的生产级框架,用于建模超长用户行为序列。首先,采用内容原生的语义ID解决表示瓶颈,通过深度截断的粗粒度语义ID将嵌入表大小压缩至语料基数以下,该紧凑表示可通过共享语义前缀自然泛化至冷启动内容。其次,引入全局感知压缩Transformer,利用无参的时间折叠与统一全局查询集成,有效压缩序列,缓解标准自注意力带来的内存与计算瓶颈。离线评估显示峰值内存下降一个数量级,计算开销显著降低。这一效率提升使生产环境可负担更长序列,大型在线A/B测试验证了用户满意度与内容消费量的显著提升。
原文摘要 · Abstract (English)
Capturing user interests across extensive watch histories is critical for short-form video recommendation, yet scaling sequence length is limited by two bottlenecks: the semantic sparsity of atomic Video IDs and the quadratic computational complexity of Transformers. Traditional orthogonal Video IDs fail to capture content relationships and demand large embedding tables, while the quadratic complexity of self-attention restricts the maximum sequence length under strict industrial latency and resource constraints. In this work, we present a production-deployed framework for modeling ultra-long user behavior sequences at a billion-user scale. We first address the representation bottleneck by adopting content-native Semantic IDs. By utilizing depth-truncated, coarse-grained Semantic IDs, we shrink the embedding table size from corpus cardinality. This compact representation naturally generalizes to cold-start content through shared semantic prefixes. Second, to overcome the sequence scaling barrier, we introduce a Global-Aware Compression Transformer that leverages non-parametric temporal folding and unified global query integration to effectively condense the sequence, alleviating both the memory and computational bottlenecks of standard self-attention. Offline profiling on our computing infrastructure demonstrates an order-of-magnitude reduction in peak memory footprint and a drastic decrease in computational overhead. This efficiency gain enables supporting longer sequence lengths at an affordable cost in production, yielding substantial online gains in satisfied user engagement and satisfied content consumption in large-scale online A/B tests.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。