解决大规模稠密检索中双塔模型的表示错位与索引不一致问题。
A Simple and Effective Framework for Symmetric Consistent Indexing in Large-Scale Dense Retrieval
- 通过输入交换机制统一双塔表示空间,零参数增量。
- 采用双视角索引策略,训练与推理路径保持一致。
- 轻量高效,支持千亿级部署,提升长尾查询效果。
稠密检索因其高效率和优异准确性,已成为大规模信息检索系统的行业标准。其核心采用粗到精的分层架构,可在百亿规模语料上实现毫秒级响应,不仅适用于传统搜索推荐场景,也支撑大语言模型驱动的生成式推荐,其中语义ID作为粗到精表示的基础。然而,广泛使用的双塔编码架构存在表示空间错位和检索索引不一致等问题,导致匹配精度下降、检索稳定性变差,尤其在语义ID生成中进一步制约下游生成模型性能上限。本文提出名为SCI的简单有效框架,包含两个协同模块:对称表示对齐模块利用创新的输入交换机制,在不增加参数的情况下统一双塔表示空间;一致索引与双塔协同模块通过双视角索引策略重构检索路径,确保训练到推理的一致性。该框架系统性强、轻量且工程友好,开销极小,完全支持千亿级部署。理论分析证明其有效性,并在公开数据集及真实电商数据集上验证了性能提升。
原文摘要 · Abstract (English)
Dense retrieval has become the industry standard in large-scale information retrieval systems due to its high efficiency and competitive accuracy. Its core relies on a coarse-to-fine hierarchical architecture that enables rapid candidate selection and precise semantic matching, achieving millisecond-level response over billion-scale corpora. This capability makes it essential not only in traditional search and recommendation scenarios but also in the emerging paradigm of generative recommendation driven by large language models, where semantic IDs-themselves a form of coarse-to-fine representation-play a foundational role. However, the widely adopted dual-tower encoding architecture introduces inherent challenges, primarily representational space misalignment and retrieval index inconsistency, which degrade matching accuracy, retrieval stability, and performance on long-tail queries. These issues are further magnified in semantic ID generation, ultimately limiting the performance ceiling of downstream generative models. To address these challenges, this paper proposes a simple and effective framework named SCI comprising two synergistic modules: a symmetric representation alignment module that employs an innovative input-swapping mechanism to unify the dual-tower representation space without adding parameters, and an consistent indexing with dual-tower synergy module that redesigns retrieval paths using a dual-view indexing strategy to maintain consistency from training to inference. The framework is systematic, lightweight, and engineering-friendly, requiring minimal overhead while fully supporting billion-scale deployment. We provide theoretical guarantees for our approach, with its effectiveness validated by results across public datasets and real-world e-commerce datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。