用测试时自适应提升方言英文任务的模型泛化能力。
Harnessing Test-time Adaptation for NLU tasks Involving Dialects of English
- 在无标注数据下，用SHOT技术实现方言间迁移。
- 方言差距越大，测试时自适应效果越明显。
- 在多数情况下，标准英语微调反而比方言数据更优。
测试时领域自适应（TTDA）是一种无需标注数据即可提升模型跨领域、跨任务和跨分布泛化能力的有效方法。在方言自然语言处理中尤为有用，因为模型通常在标准美式英语（SAE）上训练，却需在印度英语（IndE）、新加坡英语（SingE）或尼日利亚英语（NgE）等方言上评估，而这些方言与标准英语分布差异显著。由于方言数据稀缺，该方法尤为重要。本文探索了主流TTDA技术SHOT在方言NLP中的应用，在不同组合的方言GLUE数据集上进行微调与评估。结果表明，当缺乏标注数据时，SHOT是可行方案。我们还提出‘方言差距’概念，并证明其与SHOT效果呈正相关。此外发现，许多情况下在SAE上微调的表现优于在方言数据上微调。
原文摘要 · Abstract (English)
Test-time domain adaptation (TTDA) is an excellent method which helps generalize models across domains, tasks, and distributions without the use of labeled datasets. Thus, TTDA is very useful in natural language processing (NLP) in the dialectal setting, since oftentimes, models are trained on Standard American English (SAE), evaluated on Indian English (IndE), Singaporean English (SingE), or Nigerian English (NgE), of which distribution differs significantly from the former. This is especially useful since dialectal datasets are scarce. In this paper, we explore one of the most famous TTDA techniques, SHOT, in dialectal NLP. We finetune and evaluate SHOT on different combinations of dialectal GLUE. Our findings show that SHOT is a viable technique when labeled datasets are unavailable. We also theoretically propose the concept of dialectal gap and show that it has a positive correlation with the effectiveness of SHOT. We also find that in many cases, finetuning on SAE yields higher performance than finetuning on dialectal data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。