用可学习的位置核取代传统位置嵌入，提升推荐系统对序列顺序的感知能力。
Position-Aware Sequential Attention for Accurate Next Item Recommendations
- 引入可学习的位置核，在位置空间直接调节注意力权重
- 在多个基准数据集上显著优于现有强基线模型
- 适合需要精确捕捉用户行为时序模式的推荐场景
序列自注意力模型通常依赖加性位置嵌入，在输入阶段注入位置信息。当缺乏位置信号时，注意力模块对序列位置具有置换等变性，仅靠因果掩码维持时间顺序概念。我们指出，加性位置嵌入使注意力机制对顺序仅具表面敏感：位置信息与项目语义纠缠，深层传播弱，难以捕捉丰富序列模式。为此，我们提出一种核化自注意力机制，其中可学习的位置核仅在位置空间运作，与语义相似性解耦，并直接调控注意力权重。每层注意力块应用该核，实现自适应多尺度序列建模。标准下一项目推荐基准上的实验表明，本方法持续优于强基线模型。
原文摘要 · Abstract (English)
Sequential self-attention models usually rely on additive positional embeddings, which inject positional information into item representations at the input. In the absence of positional signals, the attention block is permutation-equivariant over sequence positions and thus has no intrinsic notion of temporal order beyond causal masking. We argue that additive positional embeddings make the attention mechanism only superficially sensitive to sequence order: positional information is entangled with item embedding semantics, propagates weakly in deep architectures, and limits the ability to capture rich sequential patterns. To address these limitations, we introduce a kernelized self-attention mechanism, where a learnable positional kernel operates purely in the position space, disentangled from semantic similarity, and directly modulates attention weights. When applied per attention block, this kernel enables adaptive multi-scale sequential modeling. Experiments on standard next-item prediction benchmarks show that our positional kernel attention consistently improves over strong competing baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。