用自监督对比学习提升论文引用分类效果，解决数据少、关键词干扰问题。
Adapting Pretrained Language Models for Citation Classification via Self-Supervised Contrastive Learning
- 通过句子裁剪和关键词扰动构建对比学习样本，增强模型对引用意图的捕捉能力。
- 在三个基准数据集上优于现有方法，尤其在小样本场景下提升显著。
- 兼容编码器与解码器类大模型，适配广泛预训练语言模型使用。
引用分类旨在识别学术引用背后的意图，对学术分析至关重要。以往工作通过微调预训练语言模型（PLMs）实现，利用其预训练中获得的语言知识。然而，直接微调面临标注数据稀缺、上下文噪声及虚假关键词关联等问题。本文提出新框架Citss，采用自监督对比学习缓解数据不足，并设计两种策略生成对比对：句级裁剪聚焦长上下文中的目标引用，关键词扰动减少对特定关键词的依赖。Citss不仅支持编码器型PLM，还兼容解码器型大模型，充分发挥更大规模预训练的优势。在三个基准数据集上的实验表明，相比先前最优方法，该框架表现更优。代码已开源：github.com/LITONG99/Citss。
原文摘要 · Abstract (English)
Citation classification, which identifies the intention behind academic citations, is pivotal for scholarly analysis. Previous works suggest fine-tuning pretrained language models (PLMs) on citation classification datasets, reaping the reward of the linguistic knowledge they gained during pretraining. However, directly fine-tuning for citation classification is challenging due to labeled data scarcity, contextual noise, and spurious keyphrase correlations. In this paper, we present a novel framework, Citss, that adapts the PLMs to overcome these challenges. Citss introduces self-supervised contrastive learning to alleviate data scarcity, and is equipped with two specialized strategies to obtain the contrastive pairs: sentence-level cropping, which enhances focus on target citations within long contexts, and keyphrase perturbation, which mitigates reliance on specific keyphrases. Compared with previous works that are only designed for encoder-based PLMs, Citss is carefully developed to be compatible with both encoder-based PLMs and decoder-based LLMs, to embrace the benefits of enlarged pretraining. Experiments with three benchmark datasets with both encoder-based PLMs and decoder-based LLMs demonstrate our superiority compared to the previous state of the art. Our code is available at: github.com/LITONG99/Citss
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。