用语言类型相似性聚合适配器，零成本实现多语言模型跨语言迁移。
Typologically Informed Parameter Aggregation
- 根据语言类型相似性加权聚合现有适配器，生成代理适配器。
- 在230+语言上测试，低资源语言性能提升显著，无需额外训练。
- 适合需要快速部署多语言模型的场景，尤其对无专用适配器的语言有效。
大规模多语言语言模型虽具备跨语言泛化能力，但在低资源和未见语言上表现不佳。尽管基于适配器的微调可实现参数高效，但大规模训练语言特异性适配器仍成本高昂。本文提出类型学感知参数聚合（TIPA），一种无需训练的方法，通过按语言类型相似性加权聚合已有适配器，构建代理语言适配器。将其集成至MAD-X框架后，可在不进行额外训练的情况下实现零样本跨语言迁移。我们在五个NLP任务和超过230种语言上评估了TIPA，结果表明其性能持续优于或匹配基线方法，如仅用英语微调或选择类型最相近的适配器。对于缺乏专用适配器的语言，提升最为明显。结果证明，类型学引导的聚合为无需训练的语言特异性模块提供了可行替代方案。
原文摘要 · Abstract (English)
Massively multilingual language models enable cross-lingual generalization but underperform on low-resource and unseen languages. While adapter-based fine-tuning offers a parameter-efficient solution, training language-specific adapters at scale remains costly. We introduce Typologically Informed Parameter Aggregation (TIPA), a training-free method that constructs proxy language adapters by aggregating existing ones, weighted by typological similarity. Integrated into the MAD-X framework, these proxies enable zero-shot cross-lingual transfer without additional training. We evaluate TIPA on five NLP tasks and over 230 languages. TIPA consistently outperforms or matches baselines such as English-only fine-tuning or selecting the typologically closest language adapter. We see the largest gains for languages lacking dedicated adapters. Our results demonstrate that typologically informed aggregation provides a viable alternative to language-specific modules without any training needed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。