用无标签数据自动排优先级，提升跨语言迁移效果
Model-Based Ranking of Source Languages for Zero-Shot Cross-Lingual Transfer
- 基于多语言模型隐层表示和目标语言无标注数据排序源语言
- 在POS和NER任务上平均提升35.56和18.14的NDCG指标
- 仅需25个样本即可达到92.8%性能，适合低资源场景
我们提出NN-Rank算法，用于在跨语言迁移中对源语言进行排序，该方法利用多语言模型的隐藏表示和无标签目标语言数据。实验采用两个预训练多语言模型及两个任务：词性标注（POS）和命名实体识别（NER），涵盖51个源语言，分别在56个和72个目标语言上评估。使用领域内数据时，NN-Rank超越依赖词汇与语言特征的现有基线，平均提升达35.56 NDCG（POS）和18.14 NDCG（NER）。当缺乏目标语言数据时，以往方法退化为语言层级特征，而NN-Rank仅用《圣经》这一大规模跨语言非领域语料仍保持竞争力。消融实验表明，即使仅使用25个无标签目标样本，也能生成高质量排序，达到全量数据下92.8%的NDCG性能。
原文摘要 · Abstract (English)
We present NN-Rank, an algorithm for ranking source languages for cross-lingual transfer, which leverages hidden representations from multilingual models and unlabeled target-language data. We experiment with two pretrained multilingual models and two tasks: part-of-speech tagging (POS) and named entity recognition (NER). We consider 51 source languages and evaluate on 56 and 72 target languages for POS and NER, respectively. When using in-domain data, NN-Rank beats state-of-the-art baselines that leverage lexical and linguistic features, with average improvements of up to 35.56 NDCG for POS and 18.14 NDCG for NER. As prior approaches can fall back to language-level features if target language data is not available, we show that NN-Rank remains competitive using only the Bible, an out-of-domain corpus available for a large number of languages. Ablations on the amount of unlabeled target data show that, for subsets consisting of as few as 25 examples, NN-Rank produces high-quality rankings which achieve 92.8% of the NDCG achieved using all available target data for ranking.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。