用动态融合语言模型,实现跨领域推荐的高效适配。
X-Cross: Dynamic Integration of Language Models for Cross-Domain Sequential Recommendation
- 通过低秩适配器融合多个领域语言模型,动态优化表示。
- 仅用25%额外参数,性能媲美全量微调,跨域任务精度更高。
- 适合数据稀缺场景,大幅减少微调数据与计算开销。
随着新产品每日涌现,推荐系统需快速适应新领域而无需大量重训练。本文提出X-Cross——一种新型跨领域序列推荐模型,通过集成多个领域特定语言模型实现新领域推荐;每个模型使用低秩适配器(LoRA)进行微调。给定推荐提示后,X-Cross逐层动态融合其他模型的知识,精炼各源模型的表示,并在层间传播,利用各领域适配器激活保持领域特异性,同时提升跨域适应能力。在Amazon数据集上,X-Cross性能接近全量微调的LoRA模型,但仅需25%的额外参数。在跨域任务中(如从玩具领域迁移到工具、电子或运动品类),其表现稳健,且所需微调数据比LoRA少50%-75%即可达到有效微调效果。相比其他跨域基线,X-Cross在准确率上显著提升。总体而言,该方法实现了可扩展、高效的跨域推荐,在数据受限环境下具有显著优势。
原文摘要 · Abstract (English)
As new products are emerging daily, recommendation systems are required to quickly adapt to possible new domains without needing extensive retraining. This work presents ``X-Cross'' -- a novel cross-domain sequential-recommendation model that recommends products in new domains by integrating several domain-specific language models; each model is fine-tuned with low-rank adapters (LoRA). Given a recommendation prompt, operating layer by layer, X-Cross dynamically refines the representation of each source language model by integrating knowledge from all other models. These refined representations are propagated from one layer to the next, leveraging the activations from each domain adapter to ensure domain-specific nuances are preserved while enabling adaptability across domains. Using Amazon datasets for sequential recommendation, X-Cross achieves performance comparable to a model that is fine-tuned with LoRA, while using only 25% of the additional parameters. In cross-domain tasks, such as adapting from Toys domain to Tools, Electronics or Sports, X-Cross demonstrates robust performance, while requiring about 50%-75% less fine-tuning data than LoRA to make fine-tuning effective. Furthermore, X-Cross achieves significant improvement in accuracy over alternative cross-domain baselines. Overall, X-Cross enables scalable and adaptive cross-domain recommendations, reducing computational overhead and providing an efficient solution for data-constrained environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。