arXiv:2605.10109cs.IR2026-05

让检索模型轻松理解带数字条件的查询,无需修改核心结构。

NumColBERT: Non-Intrusive Numeracy Injection for Late-Interaction Retrieval Models

  • 推理时注入数值感知机制,不改动原有检索流程。
  • 在金融、电商等场景下,数值查询准确率显著提升。
  • 适合需要快速部署且维护成本低的真实系统使用。

本文针对包含数值条件的查询(如“研发支出超过十亿美元的公司”)在密集检索中的表现难题提出解决方案。现有方法通常将查询拆分为文本与数值两部分分别评分,需修改晚交互式检索模型(如ColBERT),带来部署复杂、延迟高和维护难的问题。为此,我们提出NumColBERT,一种推理时非侵入式的数值增强方法,保持标准ColBERT索引与MaxSim评分流程不变,可直接复用现有优化与生态组件。该方法引入数值门控机制,强化关键数值约束词,抑制无关数值提及;并通过数值对比学习目标,使嵌入空间体现数值大小、单位和条件。实验表明,NumColBERT显著优于标准微调基线,性能媲美或超越依赖分离评分的先前方法,验证了非侵入式数值检索在实际部署中的可行性与可维护性。

原文摘要 · Abstract (English)

This study addresses the challenge of improving dense retrieval performance for queries containing numerical conditions, such as ``companies with more than one billion dollars in R&D expenditure.'' Although recent research has shown that standard models struggle with numeric information in domains such as finance, e-commerce, and medicine, existing solutions typically decompose queries into textual and numerical components and score them separately. These approaches modify late-interaction retrieval models such as ColBERT and introduce challenges in deployment, latency, and maintainability. To overcome these limitations, we propose NumColBERT, an inference-time non-intrusive method that enhances numerically conditioned retrieval while preserving the original late-interaction mechanism. Because NumColBERT retains the standard ColBERT indexing and MaxSim scoring pipeline, existing optimizations and ecosystem components can be reused directly, facilitating practical deployment. NumColBERT introduces a Numerical Gating Mechanism and a Numerical Contrastive Learning objective to enable numerical conditions to contribute more effectively within standard ColBERT scoring. The gating mechanism amplifies tokens carrying critical numerical constraints while suppressing context-neutral numerical mentions, and the contrastive objective shapes the embedding space to reflect numerical magnitudes, units, and conditions. Experimental results show that NumColBERT substantially outperforms standard fine-tuning baselines and achieves accuracy comparable to or better than prior approaches relying on separate textual and numerical scoring. These findings demonstrate the feasibility of numerically conditioned retrieval with a non-intrusive inference pipeline and present a maintainable solution for real-world deployment.

数值检索稀疏-密集混合检索增强非侵入式

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。