提出分层偏好感知推荐框架,提升长序列推荐效率与准确性
Beyond the Flat Sequence: Hierarchical and Preference-Aware Generative Recommendations
- 分两阶段建模:先学分层语义表示,再用偏好引导稀疏注意力
- 在工业级数据集上优于HSTU和MTGR,A/B测试显著提升点击率
- 适合处理长交互序列的推荐系统,尤其关注计算效率与信号清晰度
生成式推荐模型(如HSTU)虽能建模长用户行为序列,但其‘扁平序列’假设忽略了行为内在结构。这导致无法捕捉会话级的时间层次性,且密集注意力引入大量噪声,降低语义稀疏历史中的偏好信号质量。为此,我们提出HPGR框架,采用双阶段范式注入结构先验。第一阶段通过基于会话的掩码物品建模(MIM)预训练,学习分层且语义丰富的物品表示;第二阶段利用这些表示实现偏好引导的稀疏注意力机制,动态聚焦最相关的历史项,提升计算效率与信噪比。在APPGallery大规模工业数据集及线上A/B测试中,HPGR性能超越多个强基线,包括HSTU和MTGR。
原文摘要 · Abstract (English)
Generative Recommenders (GRs), exemplified by the Hierarchical Sequential Transduction Unit (HSTU), have emerged as a powerful paradigm for modeling long user interaction sequences. However, we observe that their "flat-sequence" assumption overlooks the rich, intrinsic structure of user behavior. This leads to two key limitations: a failure to capture the temporal hierarchy of session-based engagement, and computational inefficiency, as dense attention introduces significant noise that obscures true preference signals within semantically sparse histories, which deteriorates the quality of the learned representations. To this end, we propose a novel framework named HPGR (Hierarchical and Preference-aware Generative Recommender), built upon a two-stage paradigm that injects these crucial structural priors into the model to handle the drawback. Specifically, HPGR comprises two synergistic stages. First, a structure-aware pre-training stage employs a session-based Masked Item Modeling (MIM) objective to learn a hierarchically-informed and semantically rich item representation space. Second, a preference-aware fine-tuning stage leverages these powerful representations to implement a Preference-Guided Sparse Attention mechanism, which dynamically constrains computation to only the most relevant historical items, enhancing both efficiency and signal-to-noise ratio. Empirical experiments on a large-scale proprietary industrial dataset from APPGallery and an online A/B test verify that HPGR achieves state-of-the-art performance over multiple strong baselines, including HSTU and MTGR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。