用大模型融合多领域数据,解决无重叠用户时的推荐难题。
WeaveRec: An LLM-Based Cross-Domain Sequential Recommendation Framework with Model Merging
- 通过编织式训练多个LoRA模块,融合源域与目标域数据
- 在多场景下显著提升推荐性能,避免性能下降
- 无需增加推理开销,适合实际部署
跨领域序列推荐(CDSR)旨在通过多领域知识迁移提升用户偏好建模。现有方法多依赖重叠用户或物品建立跨域关联,但在真实场景中极少满足。大语言模型(LLM)与模型融合技术有望突破此限制,但实证表明,直接在合并数据上训练或简单融合领域专用LLM常导致性能劣于仅在目标域训练的模型。我们系统研究了该问题的根本原因,提出WeaveRec:以编织方式交叉训练多个LoRA模块,并通过模型融合整合。该框架可扩展至多源域,且不增加推理延迟或内存开销。我们还提供了理论保证,证明WeaveRec能降低目标域期望误差的上界。在单源、多源及跨平台场景下的大量实验表明,WeaveRec有效缓解性能下降,持续优于基线方法。
原文摘要 · Abstract (English)
Cross-Domain Sequential Recommendation (CDSR) seeks to improve user preference modeling by transferring knowledge from multiple domains. Despite the progress made in CDSR, most existing methods rely on overlapping users or items to establish cross-domain correlations-a requirement that rarely holds in real-world settings. The advent of large language models (LLM) and model-merging techniques appears to overcome this limitation by unifying multi-domain data without explicit overlaps. Yet, our empirical study shows that naively training an LLM on combined domains-or simply merging several domain-specific LLMs-often degrades performance relative to a model trained solely on the target domain. To address these challenges, we first experimentally investigate the cause of suboptimal performance in LLM-based cross-domain recommendation and model merging. Building on these insights, we introduce WeaveRec, which cross-trains multiple LoRA modules with source and target domain data in a weaving fashion, and fuses them via model merging. WeaveRec can be extended to multi-source domain scenarios and notably does not introduce additional inference-time cost in terms of latency or memory. Furthermore, we provide a theoretical guarantee that WeaveRec can reduce the upper bound of the expected error in the target domain. Extensive experiments on single-source, multi-source, and cross-platform cross-domain recommendation scenarios validate that WeaveRec effectively mitigates performance degradation and consistently outperforms baseline approaches in real-world recommendation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。