让检索模型在不重处理旧文档的情况下持续更新,保持对新旧内容的准确搜索。
MURR: Model Updating with Regularized Replay for Searching a Document Stream
- 用带正则化回放的策略更新模型,避免重编码旧文档
- 在流式数据中比传统方法提升检索效果与一致性
- 适合需要长期维护的实时文档检索系统
互联网持续产生新文档和用户查询,其内容随现实事件和语言演变而变化。基于固定语料训练的神经检索模型会迅速无法匹配新生成的内容和查询,导致检索失效。传统统计稀疏检索可通过更新集合统计信息适应语言变化,但神经检索模型如DPR、ColBERT的持续微调会与已编码文档产生不兼容。重新编码并重建所有历史文档索引成本高昂。本文提出MURR——一种无需重处理历史文档的神经双编码器模型更新方法,通过正则化回放机制,使模型在不重编码的前提下仍能准确检索已有文档,并持续适应最新主题。在模拟流式环境中,MURR相比其他策略,在文档与查询持续演化过程中展现出更优且更稳定的检索性能。
原文摘要 · Abstract (English)
The Internet produces a continuous stream of new documents and user-generated queries. These naturally change over time based on events in the world and the evolution of language. Neural retrieval models that were trained once on a fixed set of query-document pairs will quickly start misrepresenting newly-created content and queries, leading to less effective retrieval. Traditional statistical sparse retrieval can update collection statistics to reflect these changes in the use of language in documents and queries. In contrast, continued fine-tuning of the language model underlying neural retrieval approaches such as DPR and ColBERT creates incompatibility with previously-encoded documents. Re-encoding and re-indexing all previously-processed documents can be costly. In this work, we explore updating a neural dual encoder retrieval model without reprocessing past documents in the stream. We propose MURR, a model updating strategy with regularized replay, to ensure the model can still faithfully search existing documents without reprocessing, while continuing to update the model for the latest topics. In our simulated streaming environments, we show that fine-tuning models using MURR leads to more effective and more consistent retrieval results than other strategies as the stream of documents and queries progresses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。