arXiv:2503.02542cs.IR2025-03中稿 · SIGIR 2025 Short P…被引 7

LREA用低秩分解提升长序列用户行为建模效率,兼顾精度与响应速度。

LREA: Low-Rank Efficient Attention on Modeling Long-Term User Behaviors for CTR Prediction

  • 通过低秩矩阵分解优化注意力计算,降低复杂度。
  • 在多个数据集上超越现有方法,线上点击率提升显著。
  • 适合需要实时推荐的工业场景,尤其长序列行为建模。

随着用户历史行为数据快速增长,用户兴趣建模成为点击率(CTR)预测的关键。传统方法依赖过滤策略或低精度注意力,常导致重要信息丢失。本文提出LREA,一种新型注意力机制,利用低秩矩阵分解提升运行效率,并设计专用损失函数,在保持注意力能力的同时保障信息完整性。推理阶段采用矩阵吸收与预存储策略,有效满足在线服务的实时性要求。大量离线与线上实验表明,该方法优于当前最优模型。

原文摘要 · Abstract (English)

With the rapid growth of user historical behavior data, user interest modeling has become a prominent aspect in Click-Through Rate (CTR) prediction, focusing on learning user intent representations. However, this complexity poses computational challenges, requiring a balance between model performance and acceptable response times for online services. Traditional methods often utilize filtering techniques. These techniques can lead to the loss of significant information by prioritizing top K items based on item attributes or employing low-precision attention mechanisms. In this study, we introduce LREA, a novel attention mechanism that overcomes the limitations of existing approaches while ensuring computational efficiency. LREA leverages low-rank matrix decomposition to optimize runtime performance and incorporates a specially designed loss function to maintain attention capabilities while preserving information integrity. During the inference phase, matrix absorption and pre-storage strategies are employed to effectively meet runtime constraints. The results of extensive offline and online experiments demonstrate that our method outperforms state-of-the-art approaches.

CTR预测注意力机制低秩分解推荐系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。