通过分层对比学习提升隐式语篇关系识别效果
Constrained Multi-Layer Contrastive Learning for Implicit Discourse Relationship Recognition
- 采用标签与实例中心的对比学习，增强语义表示
- 约束高层特征对比损失小于低层，提升判别能力
- 在PDTB2.0和3.0上显著优于传统分类方法
隐式语篇关系识别（IDRR）传统上被视为分类任务。尽管使用BERT、RoBERTa等预训练语言模型，仍需复杂多层神经网络来捕捉两个语篇单元间的交互关系，导致中间层输出对不同类别判别能力不一。为此，本文提出将监督对比学习（CL）方法——标签与实例中心的对比学习——应用于表示学习，并设计一种新型的受限多层对比学习框架，强制要求高层特征的对比损失小于低层。在PDTB 2.0与PDTB 3.0数据集上的实验表明，该方法在多分类与二分类任务中均显著提升性能。
原文摘要 · Abstract (English)
Previous approaches to the task of implicit discourse relation recognition (IDRR) generally view it as a classification task. Even with pre-trained language models, like BERT and RoBERTa, IDRR still relies on complicated neural networks with multiple intermediate layers to proper capture the interaction between two discourse units. As a result, the outputs of these intermediate layers may have different capability in discriminating instances of different classes. To this end, we propose to adapt a supervised contrastive learning (CL) method, label- and instance-centered CL, to enhance representation learning. Moreover, we propose a novel constrained multi-layer CL approach to properly impose a constraint that the contrastive loss of higher layers should be smaller than that of lower layers. Experimental results on PDTB 2.0 and PDTB 3.0 show that our approach can significantly improve the performance on both multi-class classification and binary classification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。