通过融合多语言投影提升低资源语言的跨语言推理能力
Cross-lingual Representation Learning via Centroid Intervention Fusion
- 将多语言干预投影合并为共享算子，实现跨语言知识共享
- 在4个基准上平均提升3.378个百分点，尤其改善低资源语言表现
- 无需更新参数，适合快速部署于各类大模型
大型语言模型在多语言任务中表现不均，尤其对低资源语言效果较差。推理时干预通过修改前向传播中的隐藏状态来增强跨语言迁移，无需更新模型参数。现有方法通常为每对语言学习独立投影，限制可扩展性且难以共享知识。本文提出中心点干预融合（CIF），将多个多语言干预投影整合为单一语言共享算子。在多语言常识推理、自然语言推理、事实编辑和机器翻译四个基准上，CIF在四种模型骨干上平均性能优于最强的成对干预基线3.378个百分点，同时显著提升低资源语言表现。代码已开源：https://github.com/VRCMF/CIF.git。
原文摘要 · Abstract (English)
Large language models (LLMs) exhibit uneven multilingual performance, especially when dealing with low-resource languages. Inference-time intervention offers a lightweight way to improve cross-lingual transfer by modifying the hidden states produced by the LLMs during the forward pass, without updating model parameters. However, existing cross-lingual intervention methods typically learn separate projections from source to target languages, which limits scalability and prevents knowledge sharing across languages. We propose Centroid Intervention Fusion (CIF), a projection fusion framework that consolidates multiple multilingual intervention projections into a single language-shared operator. Across multilingual commonsense reasoning, natural language inference, factual editing, and machine translation benchmarks, CIF outperforms the strongest prior pairwise intervention baseline by up to +3.378 pp on average across four model backbones, while supporting performance gains for low resource languages. The code is available at https://github.com/VRCMF/CIF.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。