用自底向上的聚类方法生成唯一且保留语义结构的标识符。
Exploring Bottom-Up Clustering for Creating Semantic IDs
- 通过自底向上聚类保持嵌入空间局部结构
- 生成的语义ID在下游任务中表现更优
- 适合需要高质量标识符的检索系统
生成式检索的成功很大程度上归功于语义ID，它们相比随机项级标识符（如哈希）能更好捕捉项目语义。然而，构建语义ID的主要挑战在于确保每个标识符对应唯一产品，并保留对下游任务有价值的信息。以往工作通过添加额外码字来去重，或使用残差量化创建层次聚类。本文提出一种生成语义ID的算法，确保标识符既唯一又保留原始嵌入的结构。关键在于采用自底向上聚类以保持嵌入空间的局部结构，提升最终语义ID的聚类质量及其在下游生成式检索中的实用性。
原文摘要 · Abstract (English)
The success of generative retrieval has largely been attributed to the use of Semantic IDs, which improve over arbitrary item-level identifiers such as hashes by capturing the semantics of items. The main challenges faced when constructing Semantic IDs, however, is in mapping each identifier to a unique product and capturing information valuable to downstream tasks. Past works have appended additional codewords to de-duplicate item identifiers and utilized residual quantization to create hierarchical clusters. In this work, we present an algorithm for generating Semantic IDs that ensure the identifiers are both unique and preserve the structure of the original embedding. Key to our work is the use of bottom-up clustering to preserve local structure in the embedding space, improving the clustering quality of the resulting Semantic IDs and their utility for downstream generative retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。