arXiv:2606.17276cs.IRcs.LG2026-06

发现大模型推荐易过度记忆,提出新训练法提升长程关联理解。

On the Memorization Behavior of LLMs in Generative Recommendation: Observations, Implications, and Training Strategies

论文配图:On the Memorization Behavior of LLMs in Generative Recommendation: Observations, Implications, and Training Strategies
图 1 · 摘自论文原文
  • 设计IIRG训练策略,教模型捕捉多跳共现与语义相似关系
  • 在非单跳可预测用户上,性能显著优于传统训练方式
  • 适合想提升推荐泛化能力的研究者和工程师

生成式推荐(GR)正成为推荐系统的新方向。尽管大语言模型(LLMs)凭借预训练知识有望超越传统模型的过拟合局限,但其固有的记忆倾向可能削弱这一优势。本文聚焦一跳记忆现象,即模型推荐训练数据中紧邻后续项。结果表明,LLMs比非LLM基线更依赖此类记忆,且其性能提升的绝大部分来自这类可预测用户。我们推测,剩余用户的改进需挖掘更复杂的项目间关系。为此提出IIRG训练策略:通过多跳序列中的项目共现学习协同关系,以及基于主题相似性的语义关系。实验显示,IIRG显著优于标准下一项目预测训练,在训练时未覆盖的一跳外测试项上表现尤为突出。

原文摘要 · Abstract (English)

Generative recommendation (GR) has emerged as a promising direction for recommender systems. Recently, large language models (LLMs) have been increasingly adopted for GR, as their rich pretrained knowledge is expected to help them generalize beyond common user behavior patterns that traditional memorization-oriented baselines can capture. However, existing LLM-based GR works largely ignore LLMs' well-known tendency to memorize, which, if present in LLMs fine-tuned for GR, would restrict their utilization of pretrained knowledge. In this work, we investigate this concern by examining one-hop memorization, where a model recommends items that are direct successors of items in the training data. We show that LLMs do this more than non-LLM-based GR models-in fact, the vast majority of their gains over GR baselines are actually on users whose target items can be predicted through one-hop memorization. We intuit that improving performance on the remaining users requires LLMs to learn richer item-item relations beyond one-hop transitions. To achieve this, we propose IIRG, a novel training strategy that teaches LLMs to capture: (1) collaborative relations derived from item co-occurrences across multiple hops in user sequences, and (2) semantic relations among items with similar themes, both of which can serve as useful recommendation signals. We show that IIRG significantly improves over LLMs trained solely with standard next-item prediction, with especially large gains for users whose test items are not covered by train-time one-hop transitions.

推荐系统大模型记忆行为训练策略

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。