HyTRec通过混合注意力机制，高效精准地处理超长用户行为序列推荐。
HyTRec: A Hybrid Temporal-Aware Attention Architecture for Long Behavior Sequential Recommendation
- 分路设计：线性注意力处理历史长序列，softmax注意力聚焦近期行为。
- 在万级行为序列上实现8%以上命中率提升，推理速度保持线性。
- 适合工业级长序列推荐场景，尤其对兴趣快速变化的用户有效。
建模用户超长行为序列已成为生成式推荐的关键挑战。现有方法面临两难：线性注意力虽高效但检索精度受限于状态容量，而softmax注意力计算开销过大。为此，我们提出HyTRec，一种混合注意力架构，显式分离长期稳定偏好与短期意图波动。将大规模历史序列交由线性注意力分支处理，保留专用softmax注意力分支捕捉近期交互，从而在包含数万次交互的工业级场景中恢复高精度检索能力。为缓解线性层对快速兴趣漂移的滞后，进一步设计时序感知增量网络（TADN），动态增强新行为信号权重，同时有效抑制历史噪声。在工业级数据集上的实验表明，该模型保持线性推理速度，显著优于强基线，尤其在超长序列用户上命中率提升超过8%，兼具高效与精准。
原文摘要 · Abstract (English)
Modeling long sequences of user behaviors has emerged as a critical frontier in generative recommendation. However, existing solutions face a dilemma: linear attention mechanisms achieve efficiency at the cost of retrieval precision due to limited state capacity, while softmax attention suffers from prohibitive computational overhead. To address this challenge, we propose HyTRec, a model featuring a Hybrid Attention architecture that explicitly decouples long-term stable preferences from short-term intent spikes. By assigning massive historical sequences to a linear attention branch and reserving a specialized softmax attention branch for recent interactions, our approach restores precise retrieval capabilities within industrial-scale contexts involving ten thousand interactions. To mitigate the lag in capturing rapid interest drifts within the linear layers, we furthermore design Temporal-Aware Delta Network (TADN) to dynamically upweight fresh behavioral signals while effectively suppressing historical noise. Empirical results on industrial-scale datasets confirm the superiority that our model maintains linear inference speed and outperforms strong baselines, notably delivering over 8% improvement in Hit Rate for users with ultra-long sequences with great efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。