arXiv:2506.16698cs.LG2025-06被引 4

用压缩语义编号提升序列推荐效率,降低存储与推理开销。

SIDE: Semantic ID Embedding for effective learning from sequences

  • 用向量量化融合多源嵌入生成紧凑语义编号
  • 实现2.4倍归一化熵增益,数据量减少3倍
  • 无需参数化查表,适合大规模工业推荐系统

基于序列的推荐模型正主导工业广告推荐系统的前沿。此类系统通常处理用户历史序列,长度在10³到10⁴量级。尽管预训练模型可管理该规模嵌入,但实时预测模型面临存储与推理成本挑战。为此,我们提出一种新方法:利用向量量化(VQ)将紧凑的语义编号(SID)作为输入,替代原始嵌入集合。本方法在已有语义编号工作基础上引入三项创新:(i) 多任务VQ-VAE框架,称作VQ融合,将多个内容嵌入与类别预测融合为单一语义编号;(ii) 无参、高粒度的SID到嵌入转换技术,称为SIDE,经两组内容嵌入验证,消除对大型参数查找表的需求;(iii) 一种新型量化方法,称作离散主成分分析(DPCA),推广并增强残差量化技术。应用于大规模工业广告推荐系统后,相比传统语义编号方法,实现2.4倍归一化熵增益(NE gain)和3倍数据足迹缩减。

原文摘要 · Abstract (English)

Sequence-based recommendations models are driving the state-of-the-art for industrial ad-recommendation systems. Such systems typically deal with user histories or sequence lengths ranging in the order of O(10^3) to O(10^4) events. While adding embeddings at this scale is manageable in pre-trained models, incorporating them into real-time prediction models is challenging due to both storage and inference costs. To address this scaling challenge, we propose a novel approach that leverages vector quantization (VQ) to inject a compact Semantic ID (SID) as input to the recommendation models instead of a collection of embeddings. Our method builds on recent works of SIDs by introducing three key innovations: (i) a multi-task VQ-VAE framework, called VQ fusion that fuses multiple content embeddings and categorical predictions into a single Semantic ID; (ii) a parameter-free, highly granular SID-to-embedding conversion technique, called SIDE, that is validated with two content embedding collections, thereby eliminating the need for a large parameterized lookup table; and (iii) a novel quantization method called Discrete-PCA (DPCA) which generalizes and enhances residual quantization techniques. The proposed enhancements when applied to a large-scale industrial ads-recommendation system achieves 2.4X improvement in normalized entropy (NE) gain and 3X reduction in data footprint compared to traditional SID methods.

推荐系统向量量化序列建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。