arXiv:2601.03260cs.CEcs.CL2026-01被引 2

构建首个关系感知的科学文献检索数据集,提升AI对论文间关联的理解能力。

SciNet: Evaluating AI Agents in Relation-Aware Scientific Literature Retrieval

  • 构建包含8940个任务的SciNet数据集,支持三层次关系理解。
  • 现有检索模型在关系任务上准确率普遍低于20%。
  • 使用SciNet可使文献综述质量提升25.3%,适合科研助手研发者。

AI代理在科学文献检索中广泛应用,催生了Deep Research等工具。然而,现有检索代理主要依赖关键词或嵌入方法,虽能捕捉内容相似性,却难以理解论文间的复杂关系网络,如识别支持或矛盾研究、追踪技术脉络。这一根本局限常导致知识结构碎片化、研究情感误判及集体科学进展建模失效。为此,我们提出SciNet——首个面向信息检索代理的科学网络关系感知数据集。基于涵盖7个学科、2.69亿篇论文的元数据库,包含8,940个精心设计的任务,系统捕捉三个层次的关系理解:个体中心的新型知识结构检索、成对学术关系识别、路径式科学演化重建。对三类检索代理的广泛评估显示,其在关系感知任务上的准确率普遍低于20%,凸显当前检索范式的根本缺陷。更重要的是,在下游文献综述应用中,赋能SciNet的代理使综述质量提升25.3%,表明关系感知检索对深化科学洞察的关键价值。我们已将SciNet公开发布于https://github.com/tsinghua-fib-lab/SciNet,以支持未来研究。

原文摘要 · Abstract (English)

AI agents have seen widespread adoption in information retrieval for scientific research, giving rise to tools such as Deep Research. However, existing retrieval agents mainly rely on keyword- or embedding-based methods. While effective at capturing content-level similarities, they struggle to understand complex relational networks among scientific papers, such as identifying corroborating or conflicting studies and tracing technological lineages. This fundamental limitation often results in fragmented knowledge structures, misinterpreted research sentiment, and ineffective modeling of collective scientific progress. To address this limitation, we introduce SciNet, the first Scientific Network relation-aware dataset for information retrieval agents. Built on a meta-database of 269 million papers across 7 disciplines and containing 8,940 carefully designed tasks, SciNet systematically captures three levels of relational understanding: ego-centric retrieval of papers with novel knowledge structures, pairwise identification of scholarly relationships, and path-wise reconstruction of scientific evolution. Extensive evaluation of three categories of retrieval agents shows that their accuracy on relation-aware tasks often falls below 20%, highlighting a fundamental shortcoming of current retrieval paradigms. Importantly, in a downstream literature review application, agents empowered with SciNet achieve a 25.3% improvement in review quality, highlighting the critical value of relation-aware retrieval for deepening scientific insights. We publicly release SciNet at https://github.com/tsinghua-fib-lab/SciNet to support future research.

文献检索关系感知AI代理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。