为语义ID设计专用编码器，提升推荐系统精度
LLMs Need Encoders for Semantic IDs Too
- 用前缀记忆表构建轻量级语义ID编码器
- 在Pinterest数据上提升深度语义ID准确率46%
- 特别擅长处理贪婪解码失败的困难样本
多模态大模型使用专用编码器（如视觉编码器、音频编解码器）来捕捉非语言模态的结构信息，因为原始令牌嵌入无法表达模态特异性。我们指出，生成式推荐中使用的层次化语义ID（SIDs）也是一种类似模态：一个SID级别令牌的含义依赖于其前缀上下文，但现有系统仅将其加入词汇表，依赖训练从头学习上下文相关语义。为此，我们提出PrefixMem，一种基于前缀n-gram记忆表的轻量级SID编码器，为大模型在SID令牌位置提供结构化、前缀条件化的表示。与多模态大模型中的视觉编码器类似，PrefixMem可独立预训练后接入任意大模型进行联合训练。我们在Pinterest大规模数据上对多种大模型家族进行评估，结果显示PrefixMem在相同训练算力下使最深层语义ID准确率提升最高达46%相对，完整语义ID检索召回率提升最高22%相对。该编码器的优势集中在贪婪解码失败的困难样本上，准确率提升最高达77%相对，证实了语义ID令牌确实需要专用编码器，如同其他非语言模态一样。
原文摘要 · Abstract (English)
Multimodal LLMs use dedicated encoders to bridge non-language modalities (vision encoders for images, depth models for audio codec tokens) because raw token embeddings alone cannot capture modality-specific structure. We argue that Semantic IDs (SIDs), the hierarchical codes used in generative recommendation, constitute another such modality: a SID level token's meaning depends on its prefix context, yet current systems simply add SID tokens to the vocabulary and rely on training to learn these context-dependent meanings from scratch. We propose PrefixMem, a lightweight SID encoder based on prefix n-gram memory tables that provides the LLM with structured, prefix-conditioned representations at SID token positions. Like vision encoders in multimodal LLMs, PrefixMem can be pre-trained independently and then attached to any LLM for joint training. We evaluate on large-scale data from Pinterest across multiple LLM families and show that PrefixMem improves deepest-level SID accuracy by up to 46% relative and full-SID retrieval recall by up to 22% relative at matched training compute. The encoder's benefit concentrates on hard examples where greedy decoding fails, with up to 77% relative accuracy gains, confirming that SID tokens benefit from a dedicated encoder just as other non-language modalities do.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。