分离训练语言与任务增量，推理时组合提升低资源场景表现
DeltaMerge-LowRes: Composing Language and Task Deltas for Low-Resource Adaptation
- 分别用无标注语料和英文标注数据学习语言与任务增量
- 跨轴TIES规则使摘要任务在3/4语言上提升4-7点chrF
- 新融合策略可降低分类模型置信度偏差36%，适合低资源应用
在仅数百条标注样本的低资源NLP场景下，如何同时适配新语言和新任务是常见挑战。现有方法通常需联合微调，成本高昂。本文提出DeltaMerge-LowRes，将语言和任务两个维度解耦：从无标注单语文本中学习语言增量Δ_L，从标注英文数据中学习任务增量Δ_T，再在推理时通过四种规则（加法、激活引导、稀疏感知、跨轴TIES）进行组合。在四个非洲语言、四个任务类别共158个评估单元上，使用10,000样本成对自助法验证发现：(i) 跨轴TIES在3/4语言的摘要任务上取得+4至+7的chrF提升（最高达18.59，远超仅任务微调的13.80）；(ii) 问答任务F1提升2.32，准确率提升2.91；(iii) 稀疏感知融合使分类任务的期望校准误差（ECE）下降36%，且宏平均F1保持不变。不同融合规则显著改变模型保留、抑制与校准的内容。代码与所有日志均已公开。
原文摘要 · Abstract (English)
Adapting a multilingual encoder to a new language \emph{and} a new task with only a few hundred gold examples is a common low-resource NLP setting, yet the two axes are usually fused via an expensive language--task fine-tuning run. We ask whether they can instead be trained separately and recombined in weight space. \DeltaMergeLowRes{} learns a language delta $Δ_L$ from unlabeled monolingual text and a task delta $Δ_T$ from labeled English data, then composes them at inference under one of four rules: additive, activation-guided, sparsity-aware, and a novel \emph{cross-axis TIES}. The new rule adapts the TIES-Merging steps of trimming, sign election, and merging to the language and task axes rather than to two task axes. Holding $(Δ_L,Δ_T)$ fixed across rules on four task families and four African languages ($158$ evaluated cells, $10{,}000$-sample paired bootstrap per cell), we find: (i) cross-axis TIES wins summarisation on $3/4$ languages by $+4$ to $+7$ chrF (chrF $18.59$ vs.\ $13.80$ task-only); (ii) it improves QA F1 by $+2.32$ and EM by $+2.91$; and (iii) sparsity-aware merging cuts classification ECE by $36\%$ at parity macro-F1. The composition rule materially changes what the merged model preserves, suppresses, and calibrates. We release all JSON traces and a claim ledger.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。