arXiv:2601.15930cs.IRcs.AI2026-01被引 2

通过模型合并实现跨时空场景的生成式推荐,降低部署成本。

MMGRid: Navigating Temporal-aware and Cross-domain Generative Recommendation via Model Merging

  • 构建统一框架MMGRid,整合不同时间与领域下的生成推荐模型。
  • 发现参数冲突源于令牌分布偏移,可通过替换基模型缓解。
  • 最优合并权重与用户交互特征相关,适合实际推荐系统部署。

模型合并(MM)为在无需原始训练数据或昂贵重训练的情况下集成多个专用模型提供了高效机制。尽管在计算机视觉等领域取得成功,其在推荐系统中的应用仍基本未被探索。生成式推荐(GR)作为推荐系统的新范式,面临模型规模激增和计算开销巨大的挑战,使模型合并对成本敏感的部署尤为吸引人。本文首次从上下文视角系统研究了生成式推荐中的模型合并。聚焦于现实场景中一个基础但未被充分研究的问题:如何合并针对不同现实上下文(由用户行为随时间演化及异构应用领域引起)优化的生成推荐模型。为此,我们提出统一框架MMGRid,建立一个结构化的上下文网格,组织基于同一基础大模型(LLM)并在特定上下文数据上微调得到的检查点,形成一个真实且可控的模型空间,以系统分析不同生成推荐范式与合并算法下的模型合并效果。研究揭示若干关键洞见:首先,从大模型训练生成推荐模型会因令牌分布偏移与目标差异引入参数冲突;通过基模型替换解耦任务感知与上下文特异性参数变化可缓解此问题。其次,跨上下文增量训练会引发新近偏差,可通过加权上下文合并有效平衡。值得注意的是,最优合并权重与上下文依赖的交互特征相关,为实际部署中的权重选择提供实用指导。

原文摘要 · Abstract (English)

Model merging (MM) offers an efficient mechanism for integrating multiple specialized models without access to original training data or costly retraining. While MM has demonstrated success in domains like computer vision, its role in recommender systems (RSs) remains largely unexplored. Recently, Generative Recommendation (GR) has emerged as a new paradigm in RSs, characterized by rapidly growing model scales and substantial computational costs, making MM particularly appealing for cost-sensitive deployment scenarios. In this work, we present the first systematic study of MM in GR through a contextual lens. We focus on a fundamental yet underexplored challenge in real-world: how to merge generative recommenders specialized to different real-world contexts, arising from temporal evolving user behaviors and heterogeneous application domains. To this end, we propose a unified framework MMGRid, a structured contextual grid of GR checkpoints that organizes models trained under diverse contexts induced by temporal evolution and domain diversity. All checkpoints are derived from a shared base LLM but fine-tuned on context-specific data, forming a realistic and controlled model space for systematically analyzing MM across GR paradigms and merging algorithms. Our investigation reveals several key insights. First, training GR models from LLMs can introduce parameter conflicts during merging due to token distribution shifts and objective disparities; such conflicts can be alleviated by disentangling task-aware and context-specific parameter changes via base model replacement. Second, incremental training across contexts induces recency bias, which can be effectively balanced through weighted contextual merging. Notably, we observe that optimal merging weights correlate with context-dependent interaction characteristics, offering practical guidance for weight selection in real-world deployments.

生成推荐模型合并上下文建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。