让大模型推荐更懂细节,通过细化令牌语义提升推荐效果。
Fine-grained Semantics Integration for Large Language Model-based Recommendation
- 用关键词均值池化初始化令牌嵌入,注入语义信息。
- 在细粒度令牌层级对齐语义,提升序列生成质量。
- 适用于需要精准语义理解的个性化推荐场景。
大型语言模型(LLMs)的进展推动推荐系统从判别范式转向基于LLM的生成范式,即推荐器根据历史交互自回归生成目标项目的语义标识符(SID)序列。尽管现有基于LLM的推荐器通过将预训练语言模型对齐到SID空间取得了性能提升,但建模SID空间仍面临两大挑战:(i) 语义空白初始化:SID标记通过随机采样初始化,未引入特定语义,缺乏与预训练语言空间的显式对应;(ii) 粗粒度对齐:现有基于监督微调(SFT)的对齐任务主要关注项目级优化,忽略了SID序列中单个标记的语义。为此,我们提出TS-Rec,实现令牌级语义集成。具体包含两个关键组件:(1) 语义感知嵌入初始化(SA-Init),通过教师模型提取关键词并应用均值池化,初始化SID标记嵌入;(2) 令牌级语义对齐(TS-Align),将SID序列中的个体标记与对应项目聚类的共享语义对齐。在三个真实世界基准上的大量实验表明,TS-Rec在所有标准指标上持续优于传统和生成基线。结果证实了将细粒度语义信息融入基于LLM的生成推荐器的有效性。代码已公开于 https://anonymous.4open.science/r/TS-Rec-Review-8F99/。
原文摘要 · Abstract (English)
Recent advances in Large Language Models (LLMs) have driven a shift in recommender systems from the discriminative paradigm to the LLM-based generative paradigm, where the recommender autoregressively generates sequences of semantic identifiers (SIDs) for target items conditioned on historical interaction. While prevalent LLM-based recommenders have demonstrated performance gains by aligning pretrained LLMs between the language space and the SID space, modeling the SID space still faces two fundamental challenges: (i) Semantically Meaningless Initialization: SID tokens are initialized by random sampling without incorporating token-specific semantics, leaving them without explicit semantic correspondence to the pretrained language space; and (ii) Coarse-grained Alignment: existing SFT-based alignment tasks primarily focus on item-level optimization while overlooking the semantics of individual tokens within SID sequences. To address these challenges, we propose TS-Rec, which can integrate Token-level Semantics into LLM-based Recommenders. Specifically, TS-Rec comprises two key components: (1) Semantic-Aware Embedding Initialization (SA-Init), which initializes SID token embeddings by applying mean pooling to the pretrained embeddings of keywords extracted by a teacher model; and (2) Token-level Semantic Alignment (TS-Align), which aligns individual tokens within the SID sequence with the shared semantics of the corresponding item clusters. Extensive experiments on three real-world benchmarks demonstrate that TS-Rec consistently outperforms traditional and generative baselines across all standard metrics. These findings confirm the effectiveness of incorporating fine-grained semantic information into LLM-based generative recommenders. The implementation is available at https://anonymous.4open.science/r/TS-Rec-Review-8F99/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。