低资源语言翻译中，适配器的作用主要是正则化，而非传递语言知识。
Limited-Resource Adapters Are Regularizers, Not Linguists
- 用交叉注意力微调预训练模型，结合随机初始化适配器提升性能。
- 在三种克里奥尔语上表现优于基线，但语言相似性与性能无关。
- 适配器效果主要来自参数正则化，适合研究神经模型内在机制的读者。
跨语言迁移是提升低资源语言技术的有效策略。本文针对三种克里奥尔语（与不同语系相关），采用适配器融合与交叉注意力微调的预训练机器翻译模型，显著优于基线。然而，实验发现语言相关性与适配器性能无显著关联。更意外的是，随机初始化的适配器同样有效，表明其优势主要来自参数正则化，而非语言知识迁移。分析支持这一正则化假设。结果表明，神经语言处理的成功依赖多种因素，且并非所有方法都以直观方式利用语言知识。
原文摘要 · Abstract (English)
Cross-lingual transfer from related high-resource languages is a well-established strategy to enhance low-resource language technologies. Prior work has shown that adapters show promise for, e.g., improving low-resource machine translation (MT). In this work, we investigate an adapter souping method combined with cross-attention fine-tuning of a pre-trained MT model to leverage language transfer for three low-resource Creole languages, which exhibit relatedness to different language groups across distinct linguistic dimensions. Our approach improves performance substantially over baselines. However, we find that linguistic relatedness -- or even a lack thereof -- does not covary meaningfully with adapter performance. Surprisingly, our cross-attention fine-tuning approach appears equally effective with randomly initialized adapters, implying that the benefit of adapters in this setting lies in parameter regularization, and not in meaningful information transfer. We provide analysis supporting this regularization hypothesis. Our findings underscore the reality that neural language processing involves many success factors, and that not all neural methods leverage linguistic knowledge in intuitive ways.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。