用关键词替换数字编码，让文档标识符既语义丰富又易于搜索。
C2T-ID: Converting Semantic Codebooks to Textual Document Identifiers for Generative Search
- 先聚类生成数字编码，再用高频关键词迭代替换，提升可读性。
- 在自然问题和淘宝搜索上，性能超越纯数字、纯文本等基线方法。
- 适合需要高效精准检索的生成式搜索场景，如问答与商品推荐。
设计兼具丰富语义信息与可控搜索空间的文档标识符（docid）是生成式检索（GR）中的关键挑战。现有代码本方法通过构建层次化语义树限制生成路径，但其数值标识符无法利用大语言模型预训练的语言理解能力。相反，以文本作为docid虽具更强语义表达力，却显著扩大解码空间，易受早期错误影响。为此，我们提出C2T-ID：（i）通过层次聚类构建语义数值docid；（ii）提取高频元数据关键词，迭代将每个数值标签替换为对应簇的前K个关键词；（iii）可选的双层语义平滑步骤进一步提升流畅性。在Natural Questions与淘宝商品搜索数据集上的实验表明，C2T-ID显著优于原子编码、语义代码本及纯文本docid基线方法，验证了其在语义表达与搜索空间约束间取得良好平衡的有效性。
原文摘要 · Abstract (English)
Designing document identifiers (docids) that carry rich semantic information while maintaining tractable search spaces is a important challenge in generative retrieval (GR). Popular codebook methods address this by building a hierarchical semantic tree and constraining generation to its child nodes, yet their numeric identifiers cannot leverage the large language model's pretrained natural language understanding. Conversely, using text as docid provides more semantic expressivity but inflates the decoding space, making the system brittle to early-step errors. To resolve this trade-off, we propose C2T-ID: (i) first construct semantic numerical docid via hierarchical clustering; (ii) then extract high-frequency metadata keywords and iteratively replace each numeric label with its cluster's top-K keywords; and (iii) an optional two-level semantic smoothing step further enhances the fluency of C2T-ID. Experiments on Natural Questions and Taobao's product search demonstrate that C2T-ID significantly outperforms atomic, semantic codebook, and pure-text docid baselines, demonstrating its effectiveness in balancing semantic expressiveness with search space constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。