提出轻量级交叉编码器MICE，显著降低计算开销同时保持检索效果。
MICE: Minimal Interaction Cross-Encoders for efficient Re-ranking
- 通过剔除冗余交互，设计极简交互架构
- 在域内保留90%以上性能，域外表现更优，计算量降至2.5倍
- 适合追求高效精准排序的工业检索系统
在信息检索中，交叉编码器虽具顶尖排序效果，但推理成本高，仅适用于二级重排。现有研究主要从两个方向缓解：一是通过注意力稀疏化加速推理，二是用更复杂模型提升一级检索效果以减少重排需求。本文通过深入分析交叉编码器内部机制，识别并移除冗余交互，提出MICE（极简交互交叉编码器）新架构，在保持效果的同时大幅降低计算开销。大量实验表明，MICE在域内任务中保留了同类交叉编码器90%以上的性能，域外表现甚至超越原模型，且计算量仅为原方法的2.5倍。
原文摘要 · Abstract (English)
In Information Retrieval (IR), cross-encoders deliver state-of-the-art ranking effectiveness but have a high inference cost, limiting their use to second-stage re-rankers. Prior work has addressed this bottleneck from two largely separate directions: accelerating cross-encoder inference through attention sparsification, or improving first-stage retrieval effectiveness to alleviate the need of a re-ranker, using more complex models, e.g. late-interactions. In this work, we bridge these two directions through an in-depth analysis of cross-encoder internal mechanisms. By identifying and removing superfluous interactions, we derive MICE (Minimal Interaction Cross-Encoders), a new cross-encoder architecture that retains effectiveness while reducing computational overhead. Extensive evaluations show MICE retains most of the performances of its cross-encoder counterparts in-domain and matches or even exceeds it in out-of-domain, while reducing FLOPs down to 2.5 times.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。