解决长序列推荐中重复行为占据状态的问题,提升推荐精度与效率。
SinkRec: Mitigating Semantic State Sink in Long Sequence Recommendation with Memory-Conditioned Gated Delta Networks

- 用可学习记忆模块分离重复行为与动态转移建模
- 在多个数据集上实现更高准确率,且保持线性计算复杂度
- 适合需要高效处理长序列的工业推荐系统
线性注意力虽能以线性时间高效处理长序列推荐,但其递归状态易被重复行为模式主导,形成语义状态下沉现象。本文提出SinkRec,一种混合记忆-过渡循环架构,将协同行为模式存储与动态转移建模解耦。通过残差向量量化将局部重复模式外化为可学习条件记忆,重新注入检索到的编码,并将记忆的键值对暴露给注意力模块。进一步引入时序感知状态关系差分门控增量网络(TDGD),利用记忆净化递归写入与读取,抑制被记忆覆盖的更新并移除与记忆对齐的读出响应。该设计使重复语义从状态竞争信号转为可检索模式,让递归状态专注动态转移,缓解语义状态下沉,同时保持线性时间效率。在公开及工业数据集上的实验验证了方法的有效性与高效性。
原文摘要 · Abstract (English)
Linear attention provides an efficient backbone for long-sequence recommendation by avoiding the quadratic cost of standard Transformers, but its compressed recurrent state can be dominated by repetitive behavior patterns. We identify this phenomenon as semantic state sink, where recurring semantics over-occupy the recurrent state and bias subsequent readouts. To mitigate semantic state sink, we propose SinkRec, a hybrid memory-transition looped architecture that decouples collaborative behavioral pattern storage from dynamic transition modeling. SinkRec externalizes recurring local patterns into a learnable conditional memory through residual vector quantization, reinjects the retrieved codes, and exposes memory key-value pairs to the attention block. It further introduces Temporal-Aware State-Relation Differential Gated DeltaNet (TDGD), which uses memory to purify recurrent writing and reading by suppressing memory-covered updates and removing memory-aligned readout responses. This design turns recurring semantics from state-competing signals into memory-retrievable patterns, allowing the recurrent state to focus on dynamic transitions and alleviating semantic state sink with linear-time efficiency. Experiments on public and industrial datasets demonstrate the effectiveness and efficiency of SinkRec.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。