用轻量级用户表示条件化冻结大模型，提升推荐效率与性能
REPREC: Representation Driven Parameter-Efficient Recommendation System
- 通过MLP注入器将用户嵌入转为少量可学习软令牌
- 训练仅更新注入器，保持预训练模型不变，支持短历史训练
- 在保留94%-99%长历史性能下，训练速度提升1.5倍
大型语言模型（LLM）在序列推荐中通过输入条件化或模型适配引入协同信号。然而，现有方法常需微调LLM、增加模块、表示蒸馏或对长交互历史进行物品级条件化，导致计算和部署成本上升。本文提出REPREC，一种轻量级框架，通过紧凑的用户级表示条件化冻结的LLM。REPREC利用MLP注入器将固定大小的序列编码器嵌入映射为少量可学习的软令牌，仅训练注入器而保持预训练主干不变。大量实验表明，REPREC在不同序列编码器、LLM主干及用户活跃度下均持续提升推荐性能。其紧凑的条件化机制在训练和推理阶段均具高效率。使用短历史训练但评估时采用更长上下文，仍能保持94%-99%的全历史性能，并实现平均1.50倍的每轮训练速度提升。代码已开源。
原文摘要 · Abstract (English)
Large language models (LLMs) have been applied to sequential recommendation by incorporating collaborative signals through input conditioning or model adaptation. However, existing approaches often require LLM fine-tuning, additional architectural modules, representation distillation, or item-level conditioning over long interaction histories, increasing computational and deployment costs. We propose REPREC, a lightweight framework that conditions a frozen LLM using compact user-level representations. REPREC maps a fixed-size embedding from a frozen sequential encoder into a small set of learned soft tokens through an MLP injector, training only the injector while leaving both pretrained backbones unchanged. Our extensive experiments demonstrate that REPREC consistently improves recommendation performance across different sequential encoders, LLM backbones, and user activity levels. Its compact conditioning mechanism also makes REPREC computationally efficient during both training and inference. Moreover, training with short histories while evaluating with longer contexts retains 94--99\% of full-history performance while achieving an average $1.50\times$ per-epoch training speedup. The code is available at: https://github.com/phdbotcode/REPREC
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。