提出新型聚类量化方法，提升语义标识符生成效果
PRQ-KMeans: Projection Residual Quantization for Semantic ID Tokenization
- 通过投影残差重构，更精准地去除共享特征
- 在工业数据集上达7.4%点击率提升，MRR提升11.8%
- 适合需要高精度语义编码的推荐与检索场景
语义标识符（SIDs）将实体表示为分层标记序列，用于生成式检索与推荐。残差量化标记器通过逐级选择码字并传递残差来构建序列。我们将其视为渐进共性消除过程：每个标记捕获组内共享成分，后续标记应建模剩余差异。该视角揭示三大局限：全局共享成分耗尽首层容量、硬分配忽略邻近码字的梯度相似性、全码字减法可能导致残差沿选定码字方向残留变化。为此，我们提出后处理设置下的PRQ-KMeans：移除全局均值，采用Top-k相似加权更新中心点，并以投影残差替代全码字减法，仅移除表示向量中对应中心点的分量。在大规模工业搜索数据集及四个公开推荐基准上实验表明，PRQ-KMeans在所有评估标记器中表现最优，工业数据集上最高实现7.4%的命中率提升和11.8%的MRR提升。
原文摘要 · Abstract (English)
Semantic identifiers (SIDs) represent entities as hierarchical token sequences for generative retrieval and recommendation. Residual-quantization tokenizers construct these sequences by selecting a codeword at each level and passing a residual to the next. We view this process as progressive commonality removal: each token captures a component shared within its group, while later tokens should model the remaining differences. This view reveals three limitations: a corpus-wide shared component can consume first-level capacity, hard assignment ignores graded similarities to nearby codewords, and full-codeword subtraction can leave variation along the selected-codeword direction in the next residual. We therefore develop our solution in the post-hoc setting, where residual construction is not constrained by input reconstruction. Specifically, we propose PRQ-KMeans, which removes the global-mean component, refines centroids with Top-k similarity-weighted updates, and replaces full-codeword subtraction with a projection residual that removes each representation's selected-centroid component. Experiments on a large-scale industrial search dataset and four public recommendation benchmarks show that PRQ-KMeans achieves the strongest overall performance among the evaluated tokenizers, including gains of up to 7.4% in HitRate and 11.8% in MRR on the industrial dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。