arXiv:2409.17474cs.CLcs.LG2024-09被引 6

通过动态重加权提升文本增强数据质量,显著改善分类模型性能。

Reducing and Exploiting Data Augmentation Noise through Meta Reweighting Contrastive Learning for Text Classification

  • 结合元学习与对比学习,根据数据质量动态调整增强样本权重。
  • 在七个GLUE数据集上,文本分类准确率提升1.6%至4.4%。
  • 适用于RoBERTa-base、Text-CNN等主流模型,适合注重数据增强的研究者。

数据增强在缓解数据稀缺问题和提升模型泛化能力方面已证明有效,但增强数据的质量可能参差不齐,尤其低于原始数据。为提升文本分类任务中深度模型对增强样本的利用效率,本文提出一种新框架,融合元学习与对比学习技术,基于数据质量对增强样本进行重加权并优化其特征表示。框架内设计了新的权重依赖型入队与出队算法,有效利用样本权重信息。实验表明,该框架可与现有模型(如RoBERTa-base、Text-CNN)及增强方法(如Wordnet、Easydata)协同工作,在七个GLUE基准数据集上,相较于最优基线,文本分类器平均提升1.6%,最高达4.3%;对于RoBERTa-base编码器,平均提升1.4%,最高达4.4%。深入分析揭示了各组件的非平凡贡献。代码已公开,便于复现。

原文摘要 · Abstract (English)

Data augmentation has shown its effectiveness in resolving the data-hungry problem and improving model's generalization ability. However, the quality of augmented data can be varied, especially compared with the raw/original data. To boost deep learning models' performance given augmented data/samples in text classification tasks, we propose a novel framework, which leverages both meta learning and contrastive learning techniques as parts of our design for reweighting the augmented samples and refining their feature representations based on their quality. As part of the framework, we propose novel weight-dependent enqueue and dequeue algorithms to utilize augmented samples' weight/quality information effectively. Through experiments, we show that our framework can reasonably cooperate with existing deep learning models (e.g., RoBERTa-base and Text-CNN) and augmentation techniques (e.g., Wordnet and Easydata) for specific supervised learning tasks. Experiment results show that our framework achieves an average of 1.6%, up to 4.3% absolute improvement on Text-CNN encoders and an average of 1.4%, up to 4.4% absolute improvement on RoBERTa-base encoders on seven GLUE benchmark datasets compared with the best baseline. We present an indepth analysis of our framework design, revealing the non-trivial contributions of our network components. Our code is publicly available for better reproducibility.

文本分类数据增强对比学习元学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。