arXiv:2605.14853cs.IR2026-05

将排序与检索统一为同一问题,通过联合训练提升推荐个性化效果。

Discrimination Is Generation: Unifying Ranking and Retrieval from a Tokenizer Perspective

  • 把分词器嵌入排序模型中,实现端到端联合训练。
  • 在三个公开数据集和两个工业数据集上同时提升排序与检索性能。
  • 适合关注推荐系统生成能力与个性化优化的研究者。

语义标识符(SIDs)定义了生成式推荐的生成空间,直接决定其个性化上限。然而,现有分词器独立于检索目标进行训练,导致个性化信号与SIDs构建过程完全脱节——这是生成式检索长期落后于判别式排序的根本原因。本文重新思考SIDs的本质:排序是在物品空间寻找最大值,而检索是在标记空间寻找最大值,二者本质上是同一问题在不同粒度下的求解。基于此洞察,我们提出DIG(Discrimination Is Generation),将分词器嵌入判别式排序模型中进行端到端训练,使排序器自然演变为检索器,一次训练即可获得两个模型。DIG围绕特征分配分类体系设计:物品固有静态特征被编码为SIDs,用户-物品交叉特征(u2i)在训练过程中隐式推动码本边界向推荐决策边界靠拢,一个MLP$_\mathrm{u2t}$蒸馏模块在推理阶段近似u2i在标记层面的表示。在三个公开基准和两个工业数据集上的实验表明,DIG同时提升了排序、检索及统一检索-排序的质量。

原文摘要 · Abstract (English)

Semantic IDs (SIDs) define the generation space of generative recommendation and directly determine its personalization ceiling. However, existing tokenizers are trained independently with retrieval objectives, leaving personalization signals fully decoupled from the SID construction process -- a fundamental gap that causes generative retrieval to persistently lag behind discriminative ranking. In this paper, we rethink the essence of SIDs: \emph{ranking seeks argmax in item space while retrieval seeks argmax in token space; both are the same problem solved at different granularities.} Based on this insight, we propose \DIG (\textbf{D}iscrimination \textbf{I}s \textbf{G}eneration), which embeds the tokenizer inside a discriminative ranking model for end-to-end training -- the ranker naturally becomes a retrieval model, yielding two models from a single training run. \DIG is organized around a \emph{feature assignment taxonomy}: item-intrinsic static features are encoded into SIDs, user-item cross features (u2i) implicitly drive codebook boundaries toward recommendation decision boundaries during training, and an MLP$_\mathrm{u2t}$ distillation module approximates u2i at the token level for inference. Experiments on three public benchmarks and two industrial datasets demonstrate that \DIG simultaneously improves ranking, retrieval, and unified retrieval-ranking quality.

推荐系统生成式推荐令牌化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。