通过反事实对比学习提升密集检索模型对关键段落的敏感度,增强鲁棒性与可解释性。
Unsupervised dense retrieval with conterfactual contrastive learning
- 基于反事实分析设计无监督正则化方法,提升模型对关键段落变化的响应能力。
- 无需段落级标注即可有效提取相关段落,且对抗攻击下性能优于现有方法。
- 适合关注模型鲁棒性、可解释性的信息检索研究者与应用开发者。
从大规模文档语料中高效检索出简洁候选集仍是信息检索的核心挑战。基于Transformer和预训练语言模型的密集检索模型虽表现优异,却面临可解释性差、易受对抗攻击等问题。为此,本文提出通过增强模型对细粒度相关信号的敏感性来提升其鲁棒性:理想模型应在关键相关段落被修改时产生显著差异,而对无关段落的改动保持稳定。这种敏感性使模型在对抗攻击下仍能保持可靠结果,并支持定位真正相关的文档片段,从而提升可解释性。受因果推断与反事实分析启发,我们提出一系列基于博弈论和反事实段落的无监督正则化方法。实验表明,该方法无需依赖段落级相关性标注即可有效提取关键段落;且经过正则化的密集检索模型在对抗攻击下表现超越现有最先进方法。
原文摘要 · Abstract (English)
Efficiently retrieving a concise set of candidates from a large document corpus remains a pivotal challenge in Information Retrieval (IR). Neural retrieval models, particularly dense retrieval models built with transformers and pretrained language models, have been popular due to their superior performance. However, criticisms have also been raised on their lack of explainability and vulnerability to adversarial attacks. In response to these challenges, we propose to improve the robustness of dense retrieval models by enhancing their sensitivity of fine-graned relevance signals. A model achieving sensitivity in this context should exhibit high variances when documents' key passages determining their relevance to queries have been modified, while maintaining low variances for other changes in irrelevant passages. This sensitivity allows a dense retrieval model to produce robust results with respect to attacks that try to promote documents without actually increasing their relevance. It also makes it possible to analyze which part of a document is actually relevant to a query, and thus improve the explainability of the retrieval model. Motivated by causality and counterfactual analysis, we propose a series of counterfactual regularization methods based on game theory and unsupervised learning with counterfactual passages. Experiments show that, our method can extract key passages without reliance on the passage-level relevance annotations. Moreover, the regularized dense retrieval models exhibit heightened robustness against adversarial attacks, surpassing the state-of-the-art anti-attack methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。