将用户每次交互转化为紧凑事件令牌,提升大模型推荐效果。
An Event is Worth One Token: Event Tokenization for Industrial-scale LLM Recommendation

- 用事件快照压缩成紧凑令牌,作为新输入模态
- 在工业级榜单上超越传统方法,实现计算与质量双赢
- 令牌可跨模型迁移,适合大规模推荐系统应用
基于大模型的推荐系统虽随模型规模和序列长度扩展,但每个位置仅编码文本、语义ID或少量类别特征,忽略了每条交互中丰富的用户、物品、上下文及结果信号。在自回归建模下,这导致每一步查询能力弱,且错误随序列累积放大。本文提出以事件为中心的新范式,将每次交互表示为完整的时序快照,并识别出新的可扩展维度——快照分辨率(即每事件编码的信息量)。为高效提升分辨率,提出AMBER(通过瓶颈化事件表示进行自回归建模),将每个时序快照压缩为紧凑的事件令牌(Event Token),一种新型大模型输入模态。该表示端到端学习,而事件令牌预先计算并缓存用于服务,实现快照分辨率与实时计算解耦。在工业级排序与检索基准上,AMBER优于其他推荐范式,突破计算-质量权衡边界。在足够容量下,单一统一分词器甚至优于专用实体分词器,证明不同结构实体间的正向迁移。事件令牌还可跨架构迁移:当集成至高度优化的非大模型排序器作为服务时历史特征时,带来统计显著提升。进一步扩大事件分词器容量仍可持续改进性能。
原文摘要 · Abstract (English)
LLM-based recommendation has scaled along model capacity and sequence length, yet each position encodes only text, semantic IDs, or a few categorical features, discarding rich user, item, context, and outcome signals available at each event. Under autoregressive modeling, this yields weak queries at each position and, since each position becomes context for the next, the degradation compounds across the sequence. We propose an event-centric paradigm that represents each interaction by its full temporal snapshot, and identify a new scaling dimension we term snapshot resolution: the amount of information encoded per event. To efficiently scale snapshot resolution, we introduce AMBER (Autoregressive Modeling via Bottlenecked Event Representation), which compresses each temporal snapshot into a compact Event Token, a new LLM input modality. The representation is learned end-to-end, while Event Tokens are pre-computed and cached for serving, decoupling snapshot resolution from real-time serving compute. On industrial-scale ranking and retrieval benchmarks, AMBER advances the compute-quality Pareto frontier relative to alternative recommendation paradigms. At sufficient capacity, a single unified tokenizer even outperforms dedicated per-entity tokenizers, demonstrating positive transfer across structurally different entity types. AMBER's Event Tokens also transfer across model architectures: when integrated into a heavily optimized non-LLM ranker as serving-time historical features, they yield statistically significant improvements. Further scaling Event Tokenizer capacity provides additional improvements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。