arXiv:2509.10468cs.IRcs.AI2025-09中稿 · SIGIR 2026被引 2

让推荐系统的词嵌入同时保留预训练语义与用户上下文,提升生成式推荐效果

Learning Decomposed Contextual Token Representations from Pretrained and Collaborative Signals for Generative Recommendation

  • 通过上下文感知的组合机制动态调整词嵌入
  • 在三个真实数据集上超越现有基线模型
  • 适合需要融合预训练知识与用户行为的推荐场景

生成式推荐系统通常采用两阶段范式:先用预训练分词器将物品转换为语义标识符,再训练大语言模型通过序列到序列建模生成下一个物品。但这两个阶段优化目标不一致:分词器预训练侧重语义重建,而推荐训练侧重用户交互建模。这导致两个关键问题:(i) 静态分词不佳,固定词分配无法反映多样使用场景;(ii) 预训练语义丢失,语言模型中的预训练嵌入在用户交互训练中被覆盖。为此,我们提出统一框架DECOR(DEcomposed COntextual Token Representations),在保留预训练语义的同时增强词嵌入的适应性。DECOR引入上下文化词嵌入组合机制,根据用户交互上下文优化词表示,并采用分解式嵌入融合方法,整合预训练码本嵌入与新学习的协同嵌入。在三个真实世界数据集上的实验表明,DECOR在推荐性能上持续优于当前最先进基线。

原文摘要 · Abstract (English)

Recent advances in generative recommenders adopt a two-stage paradigm: items are first tokenized into semantic IDs using a pretrained tokenizer, and then large language models (LLMs) are trained to generate the next item via sequence-to-sequence modeling. However, these two stages are optimized for different objectives: semantic reconstruction during tokenizer pretraining versus user interaction modeling during recommender training. This objective misalignment leads to two key limitations: (i) suboptimal static tokenization, where fixed token assignments fail to reflect diverse usage contexts; and (ii) discarded pretrained semantics, where pretrained knowledge - typically from language model embeddings - is overwritten during recommender training on user interactions. To address these limitations, we propose to learn $\underline{DE}$composed $\underline{CO}$ntextual Token $\underline{R}$epresentations (DECOR), a unified framework that preserves pretrained semantics while enhancing the adaptability of token embeddings. DECOR introduces contextualized token composition to refine token embeddings based on user interaction context, and decomposed embedding fusion that integrates pretrained codebook embeddings with newly learned collaborative embeddings. Experiments on three real-world datasets demonstrate that DECOR consistently outperforms state-of-the-art baselines in recommendation performance.

生成推荐上下文嵌入预训练融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。