arXiv:2410.06173cs.CLcs.AI2024-10

通过词嵌入邻域关系增强类别标签,提升少样本文本分类性能

Manual Verbalizer Enrichment for Few-Shot Text Classification

  • 利用词向量空间中的邻近关系扩展类别标签,构建更优的verbalizer
  • 在极少量标注数据下仍达到顶尖效果,资源消耗显著降低
  • 适合少样本场景下的文本分类任务,尤其适用于数据稀缺情况

随着预训练语言模型的发展,基于提示的训练已成为自然语言处理任务中广泛采用的方法,尤其在零样本或少样本场景下表现优于传统微调。在此框架中,verbalizer 的作用至关重要,它将掩码词分布转化为最终预测。本文提出 extsc{mave},一种通过利用词语嵌入空间中的邻近关系来增强类别标签的verbalizer构建方法,用于文本分类任务。同时,我们设计了一个基准评估流程,用于在少样本学习环境下评测典型verbalizer基线方法在文档分类中的表现。实验结果表明,我们的方法在使用极少资源的情况下达到了当前最优性能,并在极端数据受限条件下尤为有效。

原文摘要 · Abstract (English)

With the continuous development of pre-trained language models, prompt-based training becomes a well-adopted paradigm that drastically improves the exploitation of models for many natural language processing tasks. Prompting also shows great performance compared to traditional fine-tuning when adapted to zero-shot or few-shot scenarios where the number of annotated data is limited. In this framework, the role of verbalizers is essential, as an interpretation from masked word distributions into output predictions. In this work, we propose \acrshort{mave}, an approach for verbalizer construction by enrichment of class labels using neighborhood relation in the embedding space of words for the text classification task. In addition, we elaborate a benchmarking procedure to evaluate typical baselines of verbalizers for document classification in few-shot learning contexts. Our model achieves state-of-the-art results while using significantly fewer resources. We show that our approach is particularly effective in cases with extremely limited supervision data.

少样本学习文本分类提示工程

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。