通过动态剪枝低信息语义令牌，显著降低生成推荐系统的计算开销。
RASTP: Representation-Aware Semantic Token Pruning for Generative Recommendation with Semantic Identifiers
- 基于表征强度与注意力中心性评估语义令牌重要性
- 在三个亚马逊数据集上训练时间减少26.7%
- 适合需要高效生成式推荐的工业场景
生成式推荐系统通常使用语义标识符（SIDs）将每个物品表示为编码语义信息的令牌序列。然而，用多个SIDs表示物品ID会显著增加输入序列长度，成为计算复杂度和内存消耗的主要因素。现有工作主要聚焦于优化注意力计算和键值缓存，而本文提出Representation-Aware Semantic Token Pruning（RASTP），直接剪除输入序列中不重要的语义令牌。具体而言，RASTP通过表征幅度衡量语义显著性，并结合累积注意力权重计算注意力中心性，综合评估令牌重要性。由于能够动态剪除低信息或无关的语义令牌，实验在三个真实世界亚马逊数据集上表明，RASTP将训练时间降低26.7%，同时保持或略微提升推荐性能。代码已开源至https://github.com/Yuzt-zju/RASTP。
原文摘要 · Abstract (English)
Generative recommendation systems typically leverage Semantic Identifiers (SIDs), which represent each item as a sequence of tokens that encode semantic information. However, representing item ID with multiple SIDs significantly increases input sequence length, which is a major determinant of computational complexity and memory consumption. While existing efforts primarily focus on optimizing attention computation and KV cache, we propose RASTP (Representation-Aware Semantic Token Pruning), which directly prunes less informative tokens in the input sequence. Specifically, RASTP evaluates token importance by combining semantic saliency, measured via representation magnitude, and attention centrality, derived from cumulative attention weights. Since RASTP dynamically prunes low-information or irrelevant semantic tokens, experiments on three real-world Amazon datasets show that RASTP reduces training time by 26.7\%, while maintaining or slightly improving recommendation performance. The code has been open-sourced at https://github.com/Yuzt-zju/RASTP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。