用混合母语样本和多任务学习提升印地-英语混用语幽默讽刺检测效果
Revealing the impact of synthetic native samples and multi-tasking strategies in Hindi-English code-mixed humour and sarcasm detection
- 将单语数据混入混用语训练集，提升模型泛化能力
- 多任务学习使幽默与讽刺识别F1分数分别提升10.67%和12.35%
- 大语言模型提示微调效果不如前两种方法，仍存改进空间
本文研究了多种策略对印地-英语混用语幽默与讽刺检测的提升效果。我们尝试了三种方法：（i）母语样本混合，将单语任务样本加入混用语训练集；（ii）多任务学习（MTL），利用语义相关的仇恨检测任务的母语与混用语样本；（iii）对超大规模多语言模型（VMLMs）进行少样本上下文提示与指令微调。实验发现：（i）添加母语样本使幽默检测F1提升6.76%，讽刺检测提升8.64%；（ii）多任务学习框架下，幽默检测F1提升10.67%，讽刺检测提升12.35%；（iii）提示与指令微调未能超越其他方法。消融实验与错误分析揭示了模型仍需改进的场景。代码已开源以支持复现。
原文摘要 · Abstract (English)
In this paper, we reported our experiments with various strategies to improve code-mixed humour and sarcasm detection. Particularly, we tried three approaches: (i) native sample mixing, (ii) multi-task learning (MTL), and (iii) prompting and instruction finetuning very large multilingual language models (VMLMs). In native sample mixing, we added monolingual task samples to code-mixed training sets. In MTL learning, we relied on native and code-mixed samples of a semantically related task (hate detection in our case). Finally, in our third approach, we evaluated the efficacy of VMLMs via few-shot context prompting and instruction finetuning. Some interesting findings we got are (i) adding native samples improved humor (raising the F1-score up to 6.76%) and sarcasm (raising the F1-score up to 8.64%) detection, (ii) training MLMs in an MTL framework boosted performance for both humour (raising the F1-score up to 10.67%) and sarcasm (increment up to 12.35% in F1-score) detection, and (iii) prompting and instruction finetuning VMLMs couldn't outperform the other approaches. Finally, our ablation studies and error analysis discovered the cases where our model is yet to improve. We provided our code for reproducibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。