让大模型更懂物品间协同关系,提升推荐精准度。
Beyong Tokens: Item-aware Attention for LLM-based Recommendation
- 设计物品感知注意力,区分物品内与物品间语义
- 在多个数据集上显著优于传统令牌注意力方法
- 适合希望改进大模型推荐效果的研究者与工程师
大型语言模型(LLM)在推荐系统中受到越来越多关注。现有基于LLM的方法通常将物品表示为令牌序列,并在这些令牌上应用注意力层生成推荐。然而,由于继承了标准注意力机制,这些方法侧重于建模令牌级关系,忽视了物品作为推荐的基本单元,难以有效捕捉物品层面的协同关系。本文重新审视令牌在LLM驱动推荐中的角色,将令牌关系分为两类:(1) 物品内令牌关系,反映物品内容语义,如名称、颜色、尺寸;(2) 物品间令牌关系,编码跨物品的协同关系。基于此,提出一种新的框架——物品感知注意力机制(IAM),包含两个互补的注意力层:(1) 物品内注意力层,限制注意力仅作用于同一物品内的令牌,建模物品内容语义;(2) 物品间注意力层,仅关注跨物品的令牌关系,捕捉物品协同关系。通过这种堆叠设计,IAM明确强调物品作为推荐的基本单位,使LLM能有效利用物品级协同关系。在多个公开数据集上的大量实验表明,该方法显著提升了LLM在个性化推荐中的表现。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have recently gained increasing attention in the field of recommendation. Existing LLM-based methods typically represent items as token sequences, and apply attention layers on these tokens to generate recommendations. However, by inheriting the standard attention mechanism, these methods focus on modeling token-level relations. This token-centric focus overlooks the item as the fundamental unit of recommendation, preventing existing methods from effectively capturing collaborative relations at the item level. In this work, we revisit the role of tokens in LLM-driven recommendation and categorize their relations into two types: (1) intra-item token relations, which present the content semantics of an item, e.g., name, color, and size; and (2) inter-item token relations, which encode collaborative relations across items. Building on these insights, we propose a novel framework with an item-aware attention mechanism (IAM) to enhance LLMs for recommendation. Specifically, IAM devises two complementary attention layers: (1) an intra-item attention layer, which restricts attention to tokens within the same item, modeling item content semantics; and (2) an inter-item attention layer, which attends exclusively to token relations across items, capturing item collaborative relations. Through this stacked design, IAM explicitly emphasizes items as the fundamental units in recommendation, enabling LLMs to effectively exploit item-level collaborative relations. Extensive experiments on several public datasets demonstrate the effectiveness of IAM in enhancing LLMs for personalized recommendation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。