解决推荐中长序列行为理解难题,提升大模型持续学习能力
Full-Stack Optimized Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation
- 通过语义检索降低行为序列异构性,提升信息提取效率
- 引入软提示增强与组件交互式LoRA,显著改善序列建模效果
- 适用于长期推荐场景,尤其适合需要连续理解用户行为的研究
本文针对推荐系统中大语言模型(LLM)在处理长期序列行为时存在的理解困境,提出ReLLaX框架,在数据、提示和参数层面实现全栈优化。数据层引入语义用户行为检索(SUBR),减少序列异构性;提示层采用软提示增强(SPA),注入协同知识并对齐物品表示;参数层提出组件全交互式LoRA(CFLoRA),增强低秩适配器的表达能力。理论分析表明,现有基于LoRA的LLM4Rec方法均为CFLoRA的退化版本,受限于组件间交互。在三个公开数据集上的实验验证了其优越性,有效缓解了长期序列行为理解难题。
原文摘要 · Abstract (English)
In this paper, we address the lifelong sequential behavior incomprehension problem in large language models (LLMs) for recommendation, where LLMs struggle to extract useful information from long user behavior sequences, even within their context limits. To tackle this, we propose ReLLaX (Retrieval-enhanced Large Language models Plus), a framework offering optimization across data, prompt, and parameter levels. At the data level, we introduce Semantic User Behavior Retrieval (SUBR) to reduce sequence heterogeneity, making it easier for LLMs to extract key information. For prompt-level enhancement, we employ Soft Prompt Augmentation (SPA) to inject collaborative knowledge, aligning item representations with recommendation tasks and improving LLMs's exploration of item relationships. Finally, at the parameter level, we propose Component Fully-interactive LoRA (CFLoRA), which enhances LoRA's expressiveness by enabling interactions between its components, allowing better capture of sequential information. Moreover, we present new perspectives to compare current LoRA-based LLM4Rec methods, i.e. from both a composite and a decomposed view. We theoretically demonstrate that the ways they employ LoRA for recommendation are degraded versions of our CFLoRA, with different constraints on atom component interactions. Extensive experiments on three public datasets demonstrate ReLLaX's superiority over existing baselines and its ability to mitigate lifelong sequential behavior incomprehension effectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。