arXiv:2510.20455cs.IR2025-10被引 2

用旋转位置编码同时建模用户行为的时间与顺序,提升推荐精度。

Rotate Both Ways: Time-and-Order RoPE for Generative Recommendation

  • 设计新型旋转位置编码,将时间与序号作为角度源直接影响查询键关系
  • 在多个公开及工业数据集上,推荐准确率显著优于现有方法
  • 结构简单易部署,适合工业级生成式推荐系统

生成式推荐系统通常基于Transformer自回归模型,从用户交互历史中预测下一个物品或行为。其效果取决于模型如何表征事件在序列中的位置(离散索引)和实际发生时间(时钟时间)。现有方法多通过学习的嵌入或相对注意力偏置注入时间信息。本文认为,若设计得当,基于RoPE的方法可成为联合建模用户行为序列时空信息的更优选择。尽管原始RoPE仅考虑标记顺序,生成式推荐需同时融合事件时间与标记索引。为此,我们提出时间与顺序旋转位置编码(TO-RoPE),一种将索引与时间作为角度来源、直接塑造查询-键几何结构的旋转位置编码家族。给出三种实现:早期融合、按维度拆分、按头拆分。在多个公开数据集及一个工业级私有数据集上的大量实验表明,TO-RoPE变体在编码时间和索引方面持续优于现有方法。这些结果将旋转嵌入确立为生成式推荐中一种简单、合理且便于部署的基础方案。

原文摘要 · Abstract (English)

Generative recommenders, typically transformer-based autoregressive models, predict the next item or action from a user's interaction history. Their effectiveness depends on how the model represents where an interaction event occurs in the sequence (discrete index) and when it occurred in wall-clock time. Prevailing approaches inject time via learned embeddings or relative attention biases. In this paper, we argue that RoPE-based approaches, if designed properly, can be a stronger alternative for jointly modeling temporal and sequential information in user behavior sequences. While vanilla RoPE in LLMs considers only token order, generative recommendation requires incorporating both event time and token index. To address this, we propose Time-and-Order RoPE (TO-RoPE), a family of rotary position embedding designs that treat index and time as angle sources shaping the query-key geometry directly. We present three instantiations: early fusion, split-by-dim, and split-by-head. Extensive experiments on both publicly available datasets and a proprietary industrial dataset show that TO-RoPE variants consistently improve accuracy over existing methods for encoding time and index. These results position rotary embeddings as a simple, principled, and deployment-friendly foundation for generative recommendation.

生成推荐旋转编码时序建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。