改进双编码器检索,用神经网络动态计算相关性得分。
Hypencoder Revisited: Reproducibility and Analysis of Non-Linear Scoring for First-Stage Retrieval
- 用超网络生成查询专属的非线性评分模型,提升相关性判断能力。
- 在多个数据集上优于传统双编码器,且搜索延迟大幅降低。
- 适合关注高效检索与模型鲁棒性的研究人员。
Hypencoder 是一种检索框架,将标准双编码器中固定的内积评分函数替换为由超网络根据上下文查询嵌入生成的查询专属神经网络(q-net),在保持查询与文档独立编码的同时实现更丰富的相关性估计。本文开展复现研究,并从三方面扩展原分析:复现结果表明,Hypencoder 在域内和跨域基准测试中均优于同等训练的双编码器基线,且提出的高效搜索算法显著降低查询延迟,性能损失极小;在困难检索任务中,其在 DL-Hard 和 FollowIR 上表现更好,但在 TREC TOT 上因检查点不兼容和微调敏感性未能完全验证。此外,我们评估了引入不同预训练编码器的效果,发现性能提升依赖于编码器类型与微调策略;对比基于 Faiss 的双编码器管道,标准双编码器在全量与高效搜索场景下仍更快;最后,在对抗鲁棒性测试中,q-net 的非线性评分未表现出持续劣势。代码已公开于 https://github.com/arneeichholtz/Hypencoder-reprod。
原文摘要 · Abstract (English)
The Hypencoder, proposed by Killingback et al., is a retrieval framework that replaces the fixed inner-product scoring function used in standard bi-encoders with a query-specific neural network (the $q$-net), whose weights are generated by a hypernetwork from the contextualized query embeddings. This design enables more expressive relevance estimation while preserving independent query and document encoding. In this work, we conduct a reproducibility study of the Hypencoder and extend the original analysis in three directions. Our reproduction confirms that the Hypencoder outperforms a similarly trained bi-encoder baseline on in-domain and out-of-domain benchmarks, and that the proposed efficient search algorithm substantially reduces query latency with minimal performance loss. On hard retrieval tasks, we find partial support: the Hypencoder outperforms the baseline on DL-Hard and FollowIR, but not on TREC TOT, where checkpoint incompatibility and fine-tuning sensitivity complicate full verification. Beyond reproduction, we investigate three extensions: (i)~integrating alternative pre-trained encoders into the Hypencoder framework, where we find that performance gains depend on the encoder and fine-tuning strategy; (ii)~comparing query latency against a Faiss-based bi-encoder pipeline, revealing that standard bi-encoder retrieval remains faster under both exhaustive and efficient search settings; and (iii)~evaluating adversarial robustness, where we find that the $q$-net's non-linear scoring does not provide a consistent robustness disadvantage over inner-product scoring. Our code is publicly available at https://github.com/arneeichholtz/Hypencoder-reprod.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。