用强化微调提升大模型推荐推理能力，让推荐更智能准确。
ReRec: Reasoning-Augmented LLM-based Recommendation Assistant via Reinforcement Fine-tuning
- 通过双图增强奖励设计，让模型更懂推荐逻辑。
- 分解推理步骤并惩罚错误，显著提升多步推理能力。
- 动态调整训练难度，适合复杂推荐场景的模型训练。
随着大模型（LLM）的发展，智能推荐助手需应对复杂查询并提供个性化、基于推理的推荐。现有基于大模型的推荐系统在多步推理方面仍存在不足，亟需增强推理能力。为此，我们提出ReRec，一种新型强化微调（RFT）框架，旨在提升大模型在复杂推荐任务中的推理能力。框架包含三个核心组件：(1) 双图增强奖励设计，融合NDCG@K等推荐指标与查询对齐、偏好对齐得分，提供细粒度奖励信号；(2) 推理感知优势估计，将大模型输出分解为推理阶段，对错误步骤进行惩罚，增强推理准确性；(3) 在线课程调度器，动态评估查询难度并组织训练流程，确保强化微调过程稳定。实验表明，ReRec优于当前最优基线，并保持指令遵循与通用知识等核心能力。代码已开源：https://github.com/jiani-huang/ReRec。
原文摘要 · Abstract (English)
With the rise of LLMs, there is an increasing need for intelligent recommendation assistants that can handle complex queries and provide personalized, reasoning-driven recommendations. LLM-based recommenders show potential but face challenges in multi-step reasoning, underscoring the need for reasoning-augmented systems. To address this gap, we propose ReRec, a novel reinforcement fine-tuning (RFT) framework designed to improve LLM reasoning in complex recommendation tasks. Our framework introduces three key components: (1) Dual-Graph Enhanced Reward Shaping, integrating recommendation metrics like NDCG@K with Query Alignment and Preference Alignment Scores to provide fine-grained reward signals for LLM optimization; (2) Reasoning-aware Advantage Estimation, which decomposes LLM outputs into reasoning segments and penalizes incorrect steps to enhance reasoning of recommendation; and (3) Online Curriculum Scheduler, dynamically assess query difficulty and organize training curriculum to ensure stable learning during RFT. Experiments demonstrate that ReRec outperforms state-of-the-art baselines and preserves core abilities like instruction-following and general knowledge. Our codes are available at https://github.com/jiani-huang/ReRec.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。