研究大模型跨任务语言迁移，发现语言资源多的更易受益
Donors and Recipients: On Asymmetric Transfer Across Tasks and Languages with Parameter-Efficient Fine-Tuning
- 用LoRA在11种语言4个任务上测试单源微调的迁移效果
- 跨语言任务迁移最有效，高资源语言和通用任务收益最大
- 适合做多语言应用开发或资源有限场景优化的开发者参考
大型语言模型在多任务和多语言上表现优异，但一项任务或语言的改进如何影响其他任务和语言尚不明确。本研究在多个开源大模型家族及规模上，通过标准化的11种语言与4个基准进行受控的LoRA微调实验。在单一任务-语言源上微调后，评估其在所有其他任务-语言目标对上的迁移效果。将迁移分为三类：(i) 匹配任务（跨语言），(ii) 匹配语言（跨任务），(iii) 跨任务（跨语言）。单源微调在三类中均带来整体正向提升，但增益显著不对称。匹配任务（跨语言）迁移最为有效且可预测，主要由目标语言决定而非模型架构。我们识别出一个稳定层级：高资源语言和广泛语义任务作为高效接收者，能吸收来自多种来源的收益，而专业化任务和低资源语言则更孤立。结果表明，有效的微调需识别并利用‘捐赠者-接收者’角色以最大化下游收益。
原文摘要 · Abstract (English)
Large language models (LLMs) perform strongly across tasks and languages, yet how improvements in one task or language affect other tasks and languages remains poorly understood. We conduct a controlled LoRA fine-tuning study across multiple open-weight LLM families and scales, using a standardised grid of 11 languages and four benchmarks. We fine-tune each model on a single task-language source and measure transfer when evaluated on all other task-language target pairs. We decompose transfer into three regimes: (i) Matched-Task (Cross-Language), (ii) Matched-Language (Cross-Task), and (iii) Cross-Task (Cross-Language). Single-source fine-tuning yields a net positive uplift across regimes, but the gains are strongly asymmetric. Matched-Task (Cross-Language) transfer emerges as the most effective and predictable regime, driven principally by the identity of the target language rather than model architecture. We identify a stable hierarchy where high-resource languages and broad semantic tasks act as efficient recipients that absorb gains from diverse sources, while specialised tasks and lower-resource languages are more isolated. These results imply that effective fine-tuning requires navigating donor-recipient roles to maximise downstream gains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。