研究发现,带大小写的模型在稀疏检索中表现更差,但转为小写后性能恢复。
To Case or Not to Case: An Empirical Study in Learned Sparse Retrieval
- 对比同模型的大小写版本,发现小写处理可显著提升检索性能
- 使用小写预处理后,大写模型性能与小写模型持平
- 适合想用最新大模型做检索的研究者参考
学习型稀疏检索(LSR)方法通过构建查询和文档的稀疏词汇表示,利用倒排索引实现高效搜索。现有方法几乎全部依赖无大小写区分的骨干模型,以减少词汇不匹配问题。然而,当前最先进的语言模型仅提供有大小写版本。尽管如此,骨干模型大小写对LSR的影响尚未被系统研究,可能威胁该方法的未来发展。为此,我们在多个数据集上系统评估了同一骨干模型的大小写配对版本,以判断其在LSR中的适用性。结果表明,采用大小写骨干模型的LSR模型性能显著低于无大小写版本;但通过将输入文本统一转为小写,性能差距可完全消除。进一步的词粒度分析显示,在小写预处理下,大小写模型几乎完全抑制了大小写词汇项,实际表现等同于无大小写模型,解释了性能恢复的原因。这一发现拓展了近期大小写模型在稀疏检索中的应用潜力,促进了更强骨干架构的集成。完整代码与实现见:https://github.com/lionisakis/Uncased-vs-cased-models-in-LSR
原文摘要 · Abstract (English)
Learned Sparse Retrieval (LSR) methods construct sparse lexical representations of queries and documents that can be efficiently searched using inverted indexes. Existing LSR approaches have relied almost exclusively on uncased backbone models, whose vocabularies exclude case-sensitive distinctions, thereby reducing vocabulary mismatch. However, the most recent state-of-the-art language models are only available in cased versions. Despite this shift, the impact of backbone model casing on LSR has not been studied, potentially posing a risk to the viability of the method going forward. To fill this gap, we systematically evaluate paired cased and uncased versions of the same backbone models across multiple datasets to assess their suitability for LSR. Our findings show that LSR models with cased backbone models by default perform substantially worse than their uncased counterparts; however, this gap can be eliminated by pre-processing the text to lowercase. Moreover, our token-level analysis reveals that, under lowercasing, cased models almost entirely suppress cased vocabulary items and behave effectively as uncased models, explaining their restored performance. This result broadens the applicability of recent cased models to the LSR setting and facilitates the integration of stronger backbone architectures into sparse retrieval. The complete code and implementation for this project are available at: https://github.com/lionisakis/Uncased-vs-cased-models-in-LSR
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。