用对比学习提升模型识别习语和隐喻语言的能力
Span Modeling for Idiomaticity and Figurative Language Detection with Span Contrastive Loss
- 结合槽位损失与跨度对比学习，优化模型对习语的检测
- 在多个数据集上达到当前最优的序列准确率
- 适合研究自然语言理解与语言模型泛化能力的学者
隐喻语言包含多种类型，其中一些是非组合性的，如习语——其整体意义不等于各词意义之和。这对基于分词和上下文嵌入的语言模型构成挑战。尽管大模型通过扩大短语词汇量部分缓解此问题，但仍需少样本提示或指令微调才能有效识别。现有最佳方法多基于BERT或LSTM微调。本文提出基于BERT和RoBERTa的模型，采用槽位损失与跨度对比学习（SCL）相结合，并引入硬负样本重加权机制，显著提升习语检测性能，在已有数据集上实现当前最优序列准确率。消融实验验证了SCL的有效性及其泛化能力。同时提出几何平均F1与序列准确率（SA）作为综合评估指标，衡量模型对跨度的感知与整体表现。
原文摘要 · Abstract (English)
The category of figurative language contains many varieties, some of which are non-compositional in nature. This type of phrase or multi-word expression (MWE) includes idioms, which represent a single meaning that does not consist of the sum of its words. For language models, this presents a unique problem due to tokenization and adjacent contextual embeddings. Many large language models have overcome this issue with large phrase vocabulary, though immediate recognition frequently fails without one- or few-shot prompting or instruction finetuning. The best results have been achieved with BERT-based or LSTM finetuning approaches. The model in this paper contains one such variety. We propose BERT- and RoBERTa-based models finetuned with a combination of slot loss and span contrastive loss (SCL) with hard negative reweighting to improve idiomaticity detection, attaining state of the art sequence accuracy performance on existing datasets. Comparative ablation studies show the effectiveness of SCL and its generalizability. The geometric mean of F1 and sequence accuracy (SA) is also proposed to assess a model's span awareness and general performance together.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。