arXiv:2411.04051cs.IR2024-11被引 2

让动态文档库的检索结果可复现,支持时间旅行式查询。

Reproducible Hybrid Time-Travel Retrieval in Evolving Corpora

  • 结合Lucene与版本化列存索引,实现可复现检索。
  • 旧查询在新文档库中仍返回相同排序结果。
  • 适合专利、医学综述等需高可复现性的场景。

在需要排名列表可复现的场景中,如从动态文档库提取子集用于下游研究、专利检索或医学系统性综述,传统排名检索因全局词频统计随文档变化而失去可复现性,导致布尔检索仍为首选。本文提出一种混合检索系统:使用Lucene实现快速检索,并结合基于列存储的版本化、带时间戳的索引,支持重新执行历史查询并获得相同排序结果,同时允许对演化文档集合(如网络归档)进行时间旅行查询,保持原始排序不变。该方法使动态文档库中的检索结果在文档增减和词频变化下依然完全可复现。

原文摘要 · Abstract (English)

There are settings in which reproducibility of ranked lists is desirable, such as when extracting a subset of an evolving document corpus for downstream research tasks or in domains such as patent retrieval or in medical systematic reviews, with high reproducibility expectations. However, as global term statistics change when documents change or are added to a corpus, queries using typical ranked retrieval models are not even reproducible for the parts of the document corpus that have not changed. Thus, Boolean retrieval frequently remains the mechanism of choice in such settings. We present a hybrid retrieval system combining Lucene for fast retrieval with a column-store-based retrieval system maintaining a versioned and time-stamped index. The latter component allows re-execution of previously posed queries resulting in the same ranked list and further allows for time-travel queries over evolving collection, as web archives, while maintaining the original ranking. Thus, retrieval results in evolving document collections are fully reproducible even when document collections and thus term statistics change.

信息检索可复现性时间旅行查询

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。