用大模型先验改进推荐系统的离线强化学习，更懂用户真实偏好。
Maximum In-Support Return Modeling for Dynamic Recommendation with Language Model Prior
- 把决策变换器的拼接过程移到推理阶段，忽略失败历史。
- 用预训练大模型初始化，结合低秩微调，仅更新少量参数。
- 在五个数据集和线上模拟中表现优于现有方法，适合冷启动场景。
基于强化学习的推荐系统（RLRS）能有效处理序列推荐任务，但在现实场景中常面临反馈数据质量差或稀疏的问题。本文提出MDT4Rec，一种基于决策变换器（DT）的离线强化学习推荐框架，解决两大挑战：从次优历史中学习，以及建模复杂用户-物品交互。首先，MDT4Rec将轨迹拼接过程从训练阶段移至动作推理阶段，使系统在必要时缩短历史上下文，从而忽略负面或失败的过往经历。其次，该框架利用预训练大语言模型（LLM）初始化DT，通过多层感知机（MLPs）替代线性嵌入层以获得更灵活的表示，并采用低秩适配（LoRA）仅高效微调部分参数。我们在五个公开数据集和一个在线模拟环境中评估了MDT4Rec，结果表明其性能超越现有方法。
原文摘要 · Abstract (English)
Reinforcement Learning-based recommender systems (RLRS) offer an effective way to handle sequential recommendation tasks but often face difficulties in real-world settings, where user feedback data can be sub-optimal or sparse. In this paper, we introduce MDT4Rec, an offline RLRS framework that builds on the Decision Transformer (DT) to address two major challenges: learning from sub-optimal histories and representing complex user-item interactions. First, MDT4Rec shifts the trajectory stitching procedure from the training phase to action inference, allowing the system to shorten its historical context when necessary and thereby ignore negative or unsuccessful past experiences. Second, MDT4Rec initializes DT with a pre-trained large language model (LLM) for knowledge transfer, replaces linear embedding layers with Multi-Layer Perceptrons (MLPs) for more flexible representations, and employs Low-Rank Adaptation (LoRA) to efficiently fine-tune only a small subset of parameters. We evaluate MDT4Rec on five public datasets and in an online simulation environment, demonstrating that it outperforms existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。