用记忆令牌压缩用户长期行为，提升生成推荐效率与精度。
Recurrent Preference Memory for Efficient Long-Sequence Generative Recommendation
- 将长序列行为压缩为紧凑记忆令牌，支持并行训练
- 推理延迟降低60%以上，内存占用减少80%，准确率更高
- 适合大规模实时推荐系统，尤其关注长时兴趣建模
生成式推荐模型通常通过全注意力机制建模用户行为，但扩展至终身序列时面临计算成本过高和随机交互噪声累积的问题。为此，我们提出Rec2PM框架，将长用户交互历史压缩为紧凑的偏好记忆令牌。不同于传统递归方法的串行训练，Rec2PM采用自参考教师强制策略：利用全局历史生成参考记忆，作为并行递归更新的监督目标，实现完全并行训练的同时保留推理时的迭代更新能力。此外，通过将记忆表示为令牌嵌入而非庞大的键值缓存，Rec2PM实现了极致存储效率。在大规模基准上的实验表明，Rec2PM显著降低推理延迟与内存开销，同时优于全序列模型的准确性。分析显示，偏好记忆充当去噪信息瓶颈，有效过滤交互噪声，捕捉稳健的长期兴趣。
原文摘要 · Abstract (English)
Generative recommendation (GenRec) models typically model user behavior via full attention, but scaling to lifelong sequences is hindered by prohibitive computational costs and noise accumulation from stochastic interactions. To address these challenges, we introduce Rec2PM, a framework that compresses long user interaction histories into compact Preference Memory tokens. Unlike traditional recurrent methods that suffer from serial training, Rec2PM employs a novel self-referential teacher-forcing strategy: it leverages a global view of the history to generate reference memories, which serve as supervision targets for parallelized recurrent updates. This allows for fully parallel training while maintaining the capability for iterative updates during inference. Additionally, by representing memory as token embeddings rather than extensive KV caches, Rec2PM achieves extreme storage efficiency. Experiments on large-scale benchmarks show that Rec2PM significantly reduces inference latency and memory footprint while achieving superior accuracy compared to full-sequence models. Analysis reveals that the Preference Memory functions as a denoising Information Bottleneck, effectively filtering interaction noise to capture robust long-term interests.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。