arXiv:2606.25147cs.IRcs.AI2026-06被引 2

用离散语义令牌建模用户,兼顾可解释性与系统兼容性。

TokenMinds: Pretrained User Tokens and Embeddings for User Understanding in Large Recommender Systems

论文配图:TokenMinds: Pretrained User Tokens and Embeddings for User Understanding in Large Recommender Systems
图 1 · 摘自论文原文
  • 基于预训练模型生成用户离散语义令牌,融合物品语义标识(SID)
  • 在亿级用户规模下验证,令牌与嵌入协同提升推荐效果
  • 适合需要可解释推荐的工业级系统,尤其跨场景推荐

工业级推荐系统中常见的稠密嵌入表示存在维度固定导致的表征局限。现有基于大语言模型生成文本用户令牌的方法虽能捕捉主题共现,但难以关联物品属性且缺乏可解释性。而基于语义标识(SID)的物品离散化已在生成式推荐中证明有效,但用户侧仍缺乏探索。本文提出 TokenMinds,将 PLUM 框架从物品检索扩展至用户建模,通过编码器-解码器结构生成兼具离散语义标签与稠密嵌入的双输出表示。该设计使用户令牌具备可解释性,并与依赖稠密向量的下游模型兼容。共享的 SID 词汇表支持长视频与短视频行为统一建模,显著降低训练与推理成本。在多个 YouTube 场景的离线实验与线上全量流量部署中,系统成功支撑百亿级用户服务,异步架构实现表征生成与评分解耦。以排序为核心任务,结果表明:在工业级规模下,基于 SID 的用户令牌具有实际可行性,且与稠密嵌入在不同推荐系统中呈现互补价值。

原文摘要 · Abstract (English)

User modeling in industrial recommender systems typically produces dense embeddings, which suffer from representational constraints inherent to fixed-dimensional vectors. An emerging alternative for discrete user representation -- using LLMs to generate text-based user tokens -- captures topical co-occurrences rather than deep sequential behavior dynamics and produces outputs that are difficult to ground to item attributes. Meanwhile, Semantic ID (SID) based item tokenization has proven effective for improving generalization in generative recommendation, yet discrete SID-based representations for users remain largely unexplored. We propose TokenMinds, an industrial-scale system that extends the PLUM framework from item retrieval to user modeling, generating both discrete SID-based user tokens and dense user embeddings via an encoder-decoder architecture adapted from pre-trained LLMs. This dual-output design provides the complementary benefits of discrete, semantically grounded user representations while maintaining compatibility with existing downstream models that rely on dense embeddings. Additionally, the shared SID vocabulary naturally extends to cross-scenario modeling: by unifying long-form and short-form video behaviors into a single model, we substantially reduce training and serving costs. We validate TokenMinds through extensive offline experiments and live launches on multiple YouTube surfaces, served on full user traffic (billions of users) via an asynchronous infrastructure that decouples representation generation from downstream scoring. Focusing on ranking as the primary downstream use case, our results confirm the practical viability of SID-based user tokens at industrial scale and demonstrate that tokens and dense embeddings provide complementary value across different production ranking systems.

用户建模离散表示推荐系统SID

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。