用谱域注意力增强大模型文本信息在推荐中的表达
SpecTran: Spectral-Aware Transformer-based Adapter for LLM-Enhanced Sequential Recommendation
- 提出谱域感知的Transformer适配器，全谱选择关键信息
- 在四个数据集上平均提升9.17%，超越强基线
- 适合需要融合文本语义的序列推荐研究者
传统序列推荐模型从用户-物品交互中学习低维物品ID嵌入，常忽略物品标题或描述等文本信息。近年来大语言模型（LLMs）的发展推动了利用高维语义嵌入编码物品文本信息的研究，并设计变换方法将其注入序列推荐模型。现有方法可分为两类：基于适配器的方法存在显著维度坍缩，信息集中于少数主导维度；基于SVD的方法过于僵硬且人为，仅保留少数主谱成分，丢弃其余丰富谱信息。为此，我们提出SpecTran，一种在谱域运行的谱感知变压器适配器，能够关注全谱以选择并聚合有信息量的组件。可学习的谱位置编码将奇异值线索作为归纳偏置，引导注意力聚焦于显著谱成分，促进嵌入维度间的多样性。在四个真实世界数据集和三种序列推荐骨干网络上，其表现持续优于强基线，平均提升9.17%。
原文摘要 · Abstract (English)
Traditional sequential recommendation (SR) models learn low-dimensional item ID embeddings from user-item interactions, often overlooking textual information such as item titles or descriptions. Recent advances in Large Language Models (LLMs) have inspired a surge of research that encodes item textual information with high-dimensional semantic embeddings, and designs transformation methods to inject such embeddings into SR models. These embedding transformation strategies can be categorized into two types, both of which exhibits notable drawbacks: 1) adapter-based methods suffer from pronounced dimension collapse, concentrating information into a few dominant dimensions; 2) SVD-based methods are rigid and manual, considering only a few principal spectral components while discarding rich information in the remaining spectrum. To address these limitations, we propose SpecTran, a spectral-aware transformer-based adapter that operates in the spectral domain, attending to the full spectrum to select and aggregates informative components. A learnable spectral-position encoding injects singular-value cues as an inductive bias, guiding attention toward salient spectral components and promoting diversity across embedding dimensions. Across four real-world datasets and three SR backbones, it consistently outperforms strong baselines, achieving an average improvement of 9.17%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。