提出轻量级隐式推理框架,提升大模型推荐效率与效果
Implicit Reasoning for Large Language Model-based Generative Recommendation

- 用隐式推理替代显式思维链,避免复杂训练流程
- 性能比传统方法高6.22%,训练节省65%显卡时长
- 适合追求高效部署的推荐系统研究者与工程师
大型语言模型(LLM)被广泛用于生成式推荐(GR),可利用预训练的世界知识。然而,如何有效调用这些知识仍不清晰。主要障碍是GR通常使用语义标识符(SIDs)表示物品,这些符号在预训练中未出现,破坏了LLM的自然语言推理接口。现有方法依赖昂贵的多阶段流水线来对齐SIDs并生成显式推理过程,但难以解释各阶段必要性。本文系统分析显式推理训练流程,揭示三大局限:世界知识表述弱化、SID与自然语言嵌入空间错位、对推理质量敏感,均损害性能。为此提出轻量级隐式推理框架PauseRec,无需获取推理轨迹或进行对齐训练,带来多项优势:相比标准显式思维链方法性能最高提升6.22%,训练成本降低最高65% GPU小时,推理速度最快提升71.3%。该方法为高效可靠的LLM推荐提供了新范式。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly adopted as backbones for Generative Recommendation (GR), promising access to pretrained world knowledge. Yet reliably invoking this knowledge for GR remains poorly understood. A key obstacle is that LLM-based GR typically represents items with Semantic IDs (SIDs), disrupting LLMs' natural-language reasoning interface because these tokens are unseen by the LLM during pretraining. Existing approaches address this with expensive multi-stage pipelines that ground SIDs and elicit explicit rationales, but offer limited insight into when and why each stage is necessary. In this work, we systematically decompose explicit reasoning training pipelines for LLM-based GR, revealing three key limitations: weakened world-knowledge verbalization, misalignment between SID and natural-language token embedding spaces, and sensitivity to rationale quality, all of which hurt explicit reasoning performance. To circumvent these issues, we propose PauseRec, a lightweight implicit reasoning paradigm tailored for GR. PauseRec is exceptionally practical, avoiding costly reasoning trace acquisition and reasoning alignment training, leading to a multitude of benefits: (1) it outperforms standard explicit CoT methods by up to 6.22%, (2) it reduces training cost by up to 65% GPU hours, and (3) it speeds up inference by up to 71.3%. These results position PauseRec as a lightweight alternative to explicit rationale generation, enabling more effective and efficient LLM-based GR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。