用纯语义方式生成不冲突的唯一索引，提升大模型推荐与检索效果。
Purely Semantic Indexing for LLM-based Generative Recommendation and Retrieval
- 通过放宽最近中心点选择，实现无非语义后缀的唯一语义索引
- 在序列推荐、商品搜索和文档检索任务中均提升整体与冷启动性能
- 方法不依赖模型，适合各类生成式推荐与检索系统
语义标识符（IDs）已被证明能有效适配大语言模型用于生成式推荐与检索。然而，现有方法常因语义相似文档被分配相同ID而产生语义冲突。为避免冲突，通常添加非语义令牌进行区分，这引入了随机性并扩大搜索空间，损害性能。本文提出纯语义索引，无需附加非语义令牌即可生成唯一且保持语义的ID。通过放松严格的最近中心点选择，我们设计了两种模型无关算法：全候选匹配（ECM）与递归残差搜索（RRS）。在序列推荐、商品搜索和文档检索任务上的大量实验表明，所提方法显著提升了整体性能与冷启动表现，验证了确保ID唯一性的有效性。
原文摘要 · Abstract (English)
Semantic identifiers (IDs) have proven effective in adapting large language models for generative recommendation and retrieval. However, existing methods often suffer from semantic ID conflicts, where semantically similar documents (or items) are assigned identical IDs. A common strategy to avoid conflicts is to append a non-semantic token to distinguish them, which introduces randomness and expands the search space, therefore hurting performance. In this paper, we propose purely semantic indexing to generate unique, semantic-preserving IDs without appending non-semantic tokens. We enable unique ID assignment by relaxing the strict nearest-centroid selection and introduce two model-agnostic algorithms: exhaustive candidate matching (ECM) and recursive residual searching (RRS). Extensive experiments on sequential recommendation, product search, and document retrieval tasks demonstrate that our methods improve both overall and cold-start performance, highlighting the effectiveness of ensuring ID uniqueness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。