arXiv:2412.17534cs.IRcs.AI2024-12EMNLP被引 6

用预训练模型生成论文引用,让引用推荐更精准。

CiteBART: Learning to Generate Citations for Local Citation Recommendation

  • 在编码器-解码器框架中预训练引用标记的重建能力。
  • 在Refseer和ArXiv数据集上超越现有方法,顶3名预测错误率仅4%。
  • 适合需要高精度引用推荐的研究者和论文写作助手开发者。

局部引用推荐(LCR)旨在为给定上下文中的引用占位符推荐一组相关论文。随着生成式方法逐渐超越传统预提取与重排序的方法,该任务迎来新突破。本文提出基于编码器-解码器架构的专用预训练策略,通过掩码作者-日期引用标记并学习重建,实现引用推荐。设计了两种变体:仅使用局部上下文的基础版本(CiteBART-Base),以及融合引用论文标题和摘要的全局版本(CiteBART-Global)。CiteBART-Global在多数基准(除小规模FullTextPeerRead外)达到当前最优性能,在较大数据集如Refseer和ArXiv上表现显著。经全面实验验证,该模型具备跨数据集泛化能力;在前3个推荐结果中,宏观幻觉率(MaHR)仅为4%;当真实引用位于前k名时,其余推荐的幻觉倾向明显降低。

原文摘要 · Abstract (English)

Local citation recommendation (LCR) suggests a set of papers for a citation placeholder within a given context. The task has evolved as generative approaches have become more promising than the traditional pre-fetch and re-rank-based state-of-the-art approaches. This paper introduces citation-specific pre-training within an encoder-decoder architecture, where author-date citation tokens are masked to learn to reconstruct them to fulfill LCR. There are two variants for this pre-training. In the local context-only base scheme (CiteBART-Base), the citation token in a local context is masked to learn to predict the citation. The global version (CiteBART-Global) extends the local context with the citing paper's title and abstract to enrich the learning signal. CiteBART-Global achieves state-of-the-art performance on LCR benchmarks except for the FullTextPeerRead dataset, which is quite small to see the advantage of generative pre-training. The effect is significant in the larger benchmarks, e.g., Refseer and ArXiv., with the Refseer benchmark-trained model emerging as the best-performing model. We perform comprehensive experiments, including an ablation study, a qualitative analysis, and a taxonomy of hallucinations with detailed statistics. Our analyses confirm that CiteBART-Global has a cross-dataset generalization capability; the macro hallucination rate (MaHR) at the top-3 predictions is 4\%, and when the ground-truth is in the top-k prediction list, the hallucination tendency in the other predictions drops significantly.

引用推荐预训练生成模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。