首个基于RAG的学术引用预测基准,帮研究者自动找对参考文献。
What Should I Cite? A RAG Benchmark for Academic Citation Prediction
- 构建多层级检索生成框架,区分粗粒度与细粒度引用预测任务。
- 涵盖7267个粗粒度和8541个细粒度实例,覆盖55.4万篇论文。
- 开源工具链支持复现,适合论文写作与智能文献推荐研究者使用。
随着网络学术出版物的快速增长,每年发表的论文数量激增,使得寻找相关前期工作愈发困难。引用预测旨在自动推荐合适参考文献,帮助学者应对不断扩大的科学文献海洋。本文提出首个集成检索增强生成(RAG)的学术引用预测基准——CiteRAG,包含多层次检索策略、专用检索器与生成器。该基准有四大贡献:(1) 设定两个不同粒度的引用预测任务,任务1为粗粒度列表级预测,任务2为细粒度位置级预测;构建包含7,267个实例的任务1数据集与8,541个实例的任务2数据集,支持检索与生成能力的全面评估。(2) 构建三层次大规模语料库,覆盖55.4万篇跨主流子领域的论文,采用增量式处理流程。(3) 提出多层级混合RAG方法,通过对比学习微调嵌入模型以捕捉复杂引用关系,并搭配专用生成模型。(4) 在多种前沿语言模型上开展广泛实验,包括闭源API、开源模型及自训练生成器,验证框架有效性。开源工具包支持可复现评估,聚焦学术文献,提供首个完整的引用预测评估框架,可作为其他科学领域的方法模板。代码与数据已公开于 https://github.com/LQgdwind/CiteRAG。
原文摘要 · Abstract (English)
With the rapid growth of Web-based academic publications, more and more papers are being published annually, making it increasingly difficult to find relevant prior work. Citation prediction aims to automatically suggest appropriate references, helping scholars navigate the expanding scientific literature. Here we present \textbf{CiteRAG}, the first comprehensive retrieval-augmented generation (RAG)-integrated benchmark for evaluating large language models on academic citation prediction, featuring a multi-level retrieval strategy, specialized retrievers, and generators. Our benchmark makes four core contributions: (1) We establish two instances of the citation prediction task with different granularity. Task 1 focuses on coarse-grained list-specific citation prediction, while Task 2 targets fine-grained position-specific citation prediction. To enhance these two tasks, we build a dataset containing 7,267 instances for Task 1 and 8,541 instances for Task 2, enabling comprehensive evaluation of both retrieval and generation. (2) We construct a three-level large-scale corpus with 554k papers spanning many major subfields, using an incremental pipeline. (3) We propose a multi-level hybrid RAG approach for citation prediction, fine-tuning embedding models with contrastive learning to capture complex citation relationships, paired with specialized generation models. (4) We conduct extensive experiments across state-of-the-art language models, including closed-source APIs, open-source models, and our fine-tuned generators, demonstrating the effectiveness of our framework. Our open-source toolkit enables reproducible evaluation and focuses on academic literature, providing the first comprehensive evaluation framework for citation prediction and serving as a methodological template for other scientific domains. Our source code and data are released at https://github.com/LQgdwind/CiteRAG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。