给文档编号加价值排序，提升电商检索效果
Beyond Semantic IDs: Encoding Business-Value Ranking into Document Identifiers for Generative Retrieval
- 将文档编号拆成语义分组和商业价值排序两部分
- 在淘宝3亿商品上实验，点击率更高，GMV提升1.06%
- 适合做电商、推荐系统等需要精准排序的场景
生成式检索（GR）将检索任务建模为序列到序列生成，通过文档标识符（DocID）进行召回，因此DocID设计直接影响检索质量。现有基于离散表示学习的方法存在固有冲突问题，且与业务优化目标不一致。为此，我们提出聚类-排序标识符（CRID），将DocID解耦为语义聚类和商业价值排序，生成无冲突标识符，并支持通过组内重排实现增量更新。我们还构建了一个分析框架，将检索增益分解为个性化偏好和统计先验泛化，揭示了语义聚类规模对两者平衡的影响。在包含超过3亿项商品的淘宝电商语料上实验表明，CRID在Top-K命中率上超越最强嵌入基线，全量上线后带来+1.06%的GMV增长。
原文摘要 · Abstract (English)
Generative Retrieval (GR) formulates retrieval as a sequence-to-sequence generation task, assigning each document a document identifier (DocID) and retrieving it through autoregressive decoding, making DocID design a critical factor in retrieval quality. However, existing schemes based on discrete representation learning suffer from inherent collision issues and create a mismatch between the DocID's encoding objective and the system's business optimization target. To address these limitations, we propose \textbf{Cluster-Ranked Identifier (CRID)}, which decouples DocID into \textit{semantic clustering} and \textit{business-value ranking}, yielding collision-free identifiers that support incremental updates via intra-cluster reranking. We further introduce an analytical framework that decomposes retrieval gains into \textit{personalized preference} and \textit{statistical prior} generalization, revealing how semantic cluster size governs the balance between the two components. Experiments on a Taobao e-commerce corpus of over 300M items show that CRID surpasses the strongest embedding-based retrieval baseline on top-K Hitrate, and delivers +1.06\% GMV in full-traffic deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。