针对中文混合检索，提出端到端优化方法提升语义匹配效果。
HyReC: Exploring Hybrid-based Retriever for Chinese
- 融合词法与稠密向量表示，增强中文检索语义统一性。
- 在C-MTEB上实现超越现有方法的平均性能提升，最高达3.2%。
- 适合需要高精度中文检索的应用场景，如智能客服、知识库查询。
基于混合检索的方法通过结合稠密向量与词法检索，在工业界因性能提升而受到广泛关注。然而，这类方法在中文检索场景中的应用仍较为有限。本文提出HyReC，一种专为中文混合检索设计的端到端优化方法。HyReC通过将词项语义联合融入表示模型，增强检索一致性；引入全局-局部感知编码器（GLAE），促进词法与稠密检索间的语义共享，同时减少相互干扰；并加入归一化模块（NM），进一步实现两种检索方式的互补优化。我们在C-MTEB中文检索基准上评估了HyReC，结果表明其显著优于现有方法，平均性能提升达3.2%。
原文摘要 · Abstract (English)
Hybrid-based retrieval methods, which unify dense-vector and lexicon-based retrieval, have garnered considerable attention in the industry due to performance enhancement. However, despite their promising results, the application of these hybrid paradigms in Chinese retrieval contexts has remained largely underexplored. In this paper, we introduce HyReC, an innovative end-to-end optimization method tailored specifically for hybrid-based retrieval in Chinese. HyReC enhances performance by integrating the semantic union of terms into the representation model. Additionally, it features the Global-Local-Aware Encoder (GLAE) to promote consistent semantic sharing between lexicon-based and dense retrieval while minimizing the interference between them. To further refine alignment, we incorporate a Normalization Module (NM) that fosters mutual benefits between the retrieval approaches. Finally, we evaluate HyReC on the C-MTEB retrieval benchmark to demonstrate its effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。