arXiv:2512.12740cs.IR2025-12KDD被引 3

用指数衰减和稀疏位置机制,让推荐模型又快又准。

FuXi-$γ$: Efficient Sequential Recommendation with Exponential-Power Temporal Encoder and Diagonal-Sparse Positional Mechanism

  • 用指数衰减建模时间间隔,连续访问内存,避免计算浪费。
  • 在四个数据集上超越现有方法,训练提速4.74倍,推理提速6.18倍。
  • 适合长序列推荐场景,尤其对实时性要求高的系统有帮助。

序列推荐旨在基于用户历史交互建模其动态偏好。近期方法多采用Transformer架构捕捉全局依赖,但常因时间编码的非连续内存访问和长序列上的密集注意力导致高计算开销。为此,我们提出FuXi-γ,一种新颖的序列推荐框架,通过合理的结构设计提升效果与效率。该框架采用仅解码器的Transformer结构,引入两项关键创新:(1) 指数幂时间编码器,借鉴艾宾浩斯遗忘曲线,使用可调指数衰减函数编码相对时间间隔,支持灵活建模短期与长期偏好,同时通过连续内存访问和纯矩阵运算保持高效;(2) 对角稀疏位置机制,基于托普利茨矩阵的对称性,采用对角滑动策略剪枝低贡献注意力块。在四个真实世界数据集上的大量实验表明,FuXi-γ在推荐质量上达到当前最优水平,同时训练速度提升最高达4.74倍,推理速度提升最高达6.18倍,是长序列推荐的实用且可扩展解决方案。代码已开源:https://github.com/Yeedzhi/FuXi-gamma。

原文摘要 · Abstract (English)

Sequential recommendation aims to model users' evolving preferences based on their historical interactions. Recent advances leverage Transformer-based architectures to capture global dependencies, but existing methods often suffer from high computational overhead, primarily due to discontinuous memory access in temporal encoding and dense attention over long sequences. To address these limitations, we propose FuXi-$γ$, a novel sequential recommendation framework that improves both effectiveness and efficiency through principled architectural design. FuXi-$γ$ adopts a decoder-only Transformer structure and introduces two key innovations: (1) An exponential-power temporal encoder that encodes relative temporal intervals using a tunable exponential decay function inspired by the Ebbinghaus forgetting curve. This encoder enables flexible modeling of both short-term and long-term preferences while maintaining high efficiency through continuous memory access and pure matrix operations. (2) A diagonal-sparse positional mechanism that prunes low-contribution attention blocks using a diagonal-sliding strategy guided by the persymmetry of Toeplitz matrix. Extensive experiments on four real-world datasets demonstrate that FuXi-$γ$ achieves state-of-the-art performance in recommendation quality, while accelerating training by up to 4.74$\times$ and inference by up to 6.18$\times$, making it a practical and scalable solution for long-sequence recommendation. Our code is available at https://github.com/Yeedzhi/FuXi-gamma.

序列推荐Transformer高效建模时间编码

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。