用随机傅里叶旋转建模时间周期，提升推荐系统表现
ClockRoPE: Random Fourier Rotations for Temporal Routine Modeling
- 基于傅里叶变换设计随机旋转机制，增强位置编码表达能力
- 在视频平台上线测试中显著提升用户留存与互动指标
- 适合需要捕捉复杂时间规律的序列推荐场景
旋转位置编码（RoPE）广泛应用于基于Transformer的大语言模型。然而，其原本为实现长程注意力衰减而设计的对数线性频率调度，限制了其在具有更复杂距离相关模式的领域（如序列推荐中的时间周期性）的应用。我们研究了一般查询/键旋转的表达能力，发现任何归一化的连续正定注意力调制函数均可通过其自身傅里叶变换诱导的随机旋转近似，我们称之为随机傅里叶旋转。基于此理论，我们提出ClockRoPE用于序列推荐中的周期性模式建模，其中旋转频率由周期性注意力调制函数导出。在线A/B测试中，ClockRoPE在关键用户参与度指标上持续提升，并已在某大型视频分享平台的生产级生成式召回系统中成功部署。
原文摘要 · Abstract (English)
Rotary Position Embedding (RoPE) has been widely adopted in transformer-based large language models. However, its log-linear frequency schedule, originally designed to produce long-term attention decay, limits its adoption in domains with more complex distance-correlation patterns, such as temporal periodicity in sequential recommendation. We investigate the expressiveness of general query/key rotations and find that any normalized continuous positive-definite attention modulation function can be approximated by random rotations induced by its own Fourier transform, which we term Random Fourier Rotations. Building on this theory, we propose ClockRoPE for routine modeling in sequential recommendation, where rotation frequencies are derived from periodic attention modulation functions. In online A/B tests, ClockRoPE demonstrates consistent improvements in valued engagement metrics, and has been successfully deployed in production-scale generative retrieval system at a major video-sharing platform.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。