优化推荐系统的思维链,让小模型更高效地学习大模型的推理过程。
SCOReD: Student-Aware CoT Optimization for Recommendation Distillation

- 根据学生注意力动态调整教师思维链的每段内容,选择保留、重写等操作。
- 在推荐任务上提升1.56% NDCG、1.9% Recall@5,推理长度减少27.3%。
- 适合需要高效蒸馏推荐系统思维链的模型开发者和研究者。
推荐领域的思维链(CoT)蒸馏是强化学习训练的必要前提,但原始教师轨迹并不适合此任务。大型教师模型在推荐任务中表现出异常高的推理不确定性,反复验证答案却从不修正;在此类轨迹上进行监督微调会导致学生模型产生冗长且不修正初始猜测的输出。此外,由于推荐领域新颖,教师推理轨迹对小型学生大模型而言高度分布外。为此,我们提出学生感知的推荐蒸馏思维链优化框架(SCOReD):首先将每条教师轨迹解析为带类型片段,利用学生模型注意力评分各片段重要性;随后基于输出长度与答案对比对数概率提升,动态决定每段的编辑操作(保留/重写/合并/删除)。因此,SCOReD 在保留信息密集段的同时剔除冗余部分,并使原始教师轨迹适配学生输出分布。在经过优化后的思维链上训练的学生模型获得更清晰的学习信号,在NDCG上优于基线1.56%,Recall@5提升1.9%,同时推理长度减少27.3%。
原文摘要 · Abstract (English)
Chain-of-thought (CoT) distillation in the recommendation domain is a necessary precursor to RL training, but raw teacher traces are ill-suited to this task. Large teachers approach the recommendation task with unusually high reasoning uncertainty, repeatedly rechecking their answers without revising them; supervised fine-tuning on such traces produces verbose students that never revise their initial guess. Furthermore, due to the novelty of the recommendation domain, the teacher's reasoning traces are highly out-of-distribution for the small student LLM. We propose Student-Aware CoT Optimization for Recommendation Distillation (SCOReD), a CoT optimization framework tailored to recommendation that first parses each teacher trace into typed segments and uses the student LLM's attention to score the importance of each segment. Then SCOReD dynamically selects a per-segment edit (KEEP / REWRITE / FUSE / PRUNE) based on the output length and comparative log probability lift of the answer given the edit as per the student. Therefore, SCOReD prunes redundant sections of the reasoning trace while preserving information-dense sections and adapts raw teacher traces to the student's output distribution. Training on SCOReD-optimized CoTs provides a cleaner learning signal to the student model and improves over baseline SFT by 1.56% NDCG and 1.9% Recall@5, while reducing reasoning length by 27.3%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。