arXiv:2603.10369cs.IRcs.AI2026-03被引 2

提出新模型显式建模物品与行为的因果关系,提升推荐效率和准确率。

Beyond Interleaving: Causal Attention Reformulations for Generative Recommender Systems

  • 用注意力机制显式建模物品到行为的因果依赖,替代传统交错序列
  • 在真实社交网络数据上,损失降低0.29%~0.80%,训练快12%~23%
  • 适合关注生成式推荐系统效率与可解释性的研究者和工程师

生成式推荐系统(GR)将用户行为建模为序列生成任务,通过交错物品与行为标记实现。然而,该方法存在结构与计算低效问题:序列长度翻倍、计算开销呈二次增长,且依赖隐式注意力恢复物品与行为间的因果关系。同时,交错异质标记迫使Transformer分离语义不兼容信号,增加注意力噪声,降低表征效率。本文提出一种基于因果结构与注意力理论的生成推荐新范式。我们证明现有交错机制仅为相似性加权行为聚合的低效代理。为此,提出两种新架构:基于注意力的延迟融合行为(AttnLFA)与基于注意力的混合值聚合(AttnMVP),消除交错依赖,使序列复杂度降低50%。两者均显式编码$i_n \rightarrow a_n$的因果依赖,同时保留Transformer的表达能力。在某大型社交网络的产品推荐数据集上评估显示,AttnLFA与AttnMVP持续优于交错基线,评估损失分别降低0.29%与0.80%,并显著提升归一化熵(NE)。关键的是,性能提升伴随23%与12%的训练时间减少。结果表明,显式建模物品-行为因果关系是可扩展高效生成排序的更优设计范式。

原文摘要 · Abstract (English)

Generative Recommender Systems (GR) increasingly model user behavior as a sequence generation task by interleaving item and action tokens. While effective, this formulation introduces significant structural and computational inefficiencies: it doubles sequence length, incurs quadratic overhead, and relies on implicit attention to recover the causal relationship between an item and its associated action. Furthermore, interleaving heterogeneous tokens forces the Transformer to disentangle semantically incompatible signals, leading to increased attention noise and reduced representation efficiency.In this work, we propose a principled reformulation of generative recommendation that aligns sequence modeling with underlying causal structures and attention theory. We demonstrate that current interleaving mechanisms act as inefficient proxies for similarity-weighted action pooling. To address this, we introduce two novel architectures that eliminate interleaved dependencies to reduce sequence complexity by 50%: Attention-based Late Fusion for Actions (AttnLFA) and Attention-based Mixed Value Pooling (AttnMVP). These models explicitly encode the $i_n \rightarrow a_n$ causal dependency while preserving the expressive power of Transformer-based sequence modeling.We evaluate our framework on large-scale product recommendation data from a major social network. Experimental results show that AttnLFA and AttnMVP consistently outperform interleaved baselines, achieving evaluation loss improvements of 0.29% and 0.80%, and significant gains in Normalized Entropy (NE). Crucially, these performance gains are accompanied by training time reductions of 23% and 12%, respectively. Our findings suggest that explicitly modeling item-action causality provides a superior design paradigm for scalable and efficient generative ranking.

生成推荐因果建模注意力机制序列优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。