用线性时间余弦注意力提升推荐系统效率，大幅降低内存与耗时。
On the Efficiency of Sequentially Aware Recommender Systems: Cotten4Rec
- 采用单个优化CUDA核实现线性时间余弦相似度注意力
- 在中等序列长度下内存减少67%，运行时间降低58%
- 适合资源受限的大规模推荐场景，兼顾效率与精度
顺序推荐（SR）模型通过建模用户历史行为预测下一步交互。基于Transformer的SR方法如BERT4Rec虽能有效捕捉行为模式，但因Softmax注意力带来大量中间计算，导致显著计算开销。本文提出Cotten4Rec，采用线性时间余弦相似度注意力，通过单一优化CUDA核实现。该设计最小化中间缓冲区与内核启动开销，在中等序列长度和词汇量的数据集上，相比BERT4Rec和线性注意力基线LinRec显著降低资源消耗。在三个基准数据集上的评估表明，Cotten4Rec在内存和运行时间上均有显著减少，且推荐精度损失极小，证明其在计算资源敏感的实际大规模推荐场景中的可行性。
原文摘要 · Abstract (English)
Sequential recommendation (SR) models predict a user's next interaction by modeling their historical behaviors. Transformer-based SR methods, notably BERT4Rec, effectively capture these patterns but incur significant computational overhead due to extensive intermediate computations associated with Softmax-based attention. We propose Cotten4Rec, a novel SR model utilizing linear-time cosine similarity attention, implemented through a single optimized compute unified device architecture (CUDA) kernel. By minimizing intermediate buffers and kernel-launch overhead, Cotten4Rec substantially reduces resource usage compared to BERT4Rec and the linear-attention baseline, LinRec, especially for datasets with moderate sequence lengths and vocabulary sizes. Evaluations across three benchmark datasets confirm that Cotten4Rec achieves considerable reductions in memory and runtime with minimal compromise in recommendation accuracy, demonstrating Cotten4Rec's viability as an efficient alternative for practical, large-scale sequential recommendation scenarios where computational resources are critical.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。