让大模型自己教自己做推荐,既准又快。
SelfDR: Self-Distillation from Reasoning for LLM-Based Recommendation

- 用大模型自生成推理过程,再从中提炼推荐结果。
- 在三个数据集上准确率提升,推理成本大幅降低。
- 适合想高效部署大模型推荐系统的研究人员。
大语言模型(LLM)已成为推荐系统的重要骨干。为更好发挥其能力,推理机制被广泛引入以解析丰富的文本信号,提升推荐精度。然而,显式生成中间推理轨迹通常带来巨大计算开销,限制了在真实推荐系统中的实际部署。为此,我们提出SelfDR——一种基于推理自蒸馏的LLM推荐框架。SelfDR将大模型自身的推理增强型预测结果进行蒸馏,直接生成推荐,兼顾推荐效果与推理效率。框架所有组件均基于同一基础大模型,无需依赖外部模型。具体而言,教师推荐器通过以下游性能为奖励训练推理模块,生成有针对性的推理内容并融入输入;学生推荐器则以相同模型架构,通过动态加权策略从教师中进行自蒸馏学习。在三个公开数据集上的大量实验验证了SelfDR的有效性、合理性与高效性。代码已开源:https://github.com/JiangDeccc/SelfDistillation。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have recently emerged as powerful backbones for recommendation. To better elicit their capabilities, reasoning has been widely incorporated to help LLMs interpret rich textual signals and improve recommendation accuracy. However, explicitly generating intermediate reasoning traces often incurs substantial computational costs, which limits practical deployment in real-world recommender systems. To address this challenge, we propose SelfDR, a Self-Distillation from Reasoning framework for LLM-based Recommendation. SelfDR distills an LLM's own reasoning-enhanced predictions to produce recommendations directly, improving recommendation effectiveness while maintaining inference efficiency. All components in the framework are built on the same base LLM, without relying on any external models. Specifically, the teacher recommender is constructed by training a reasoner with downstream performance as the reward, enabling it to generate targeted rationales that are later incorporated into the teacher's input. A student recommender for direct recommendation, with the same underlying model, then learns from the teacher through self-distillation with a dynamic weighting strategy. Extensive experiments on three public datasets validate the effectiveness, rationality, and efficiency of SelfDR. Codes are available at https://github.com/JiangDeccc/SelfDistillation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。