用扩散语言模型生成情感关键词,提升低资源情感分类效果
An Effective Deployment of Diffusion LM for Data Augmentation in Low-Resource Sentiment Classification
- 通过重建情感关键词生成新样本,保持语义一致性
- 在少样本和跨域场景下准确率提升5.2%~8.7%(相对基线)
- 适合数据稀缺、情绪词关键的情感分析任务
情感分类常面临低资源挑战,如领域特定语境、标签分布不均和少样本情形。扩散语言模型在文本数据增强方面的潜力尚未被探索,且现有方法难以平衡新样本的多样性与一致性。多数方法仅对非关键词进行重述或逻辑修改。但在情感分类中,强烈情感词对整体语义至关重要。因此,我们提出DiffusionCLS,利用扩散语言模型捕捉领域知识,通过重构关键标签相关词生成伪样本,兼顾一致性和多样性,避免引入噪声并增强数据集核心特征。该方法还包含抗噪训练目标以提升泛化能力。实验表明,该方法在多种低资源场景(包括领域特异性和通用性问题)中均有效。消融实验验证了各模块有效性,可视化研究揭示了最优部署条件,进一步支持结论。
原文摘要 · Abstract (English)
Sentiment classification (SC) often suffers from low-resource challenges such as domain-specific contexts, imbalanced label distributions, and few-shot scenarios. The potential of the diffusion language model (LM) for textual data augmentation (DA) remains unexplored, moreover, textual DA methods struggle to balance the diversity and consistency of new samples. Most DA methods either perform logical modifications or rephrase less important tokens in the original sequence with the language model. In the context of SC, strong emotional tokens could act critically on the sentiment of the whole sequence. Therefore, contrary to rephrasing less important context, we propose DiffusionCLS to leverage a diffusion LM to capture in-domain knowledge and generate pseudo samples by reconstructing strong label-related tokens. This approach ensures a balance between consistency and diversity, avoiding the introduction of noise and augmenting crucial features of datasets. DiffusionCLS also comprises a Noise-Resistant Training objective to help the model generalize. Experiments demonstrate the effectiveness of our method in various low-resource scenarios including domain-specific and domain-general problems. Ablation studies confirm the effectiveness of our framework's modules, and visualization studies highlight optimal deployment conditions, reinforcing our conclusions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。