不费资源提升低语种大模型表现，推理时对齐高语种表征。
Bridging the Language Gaps in Large Language Models with Inference-Time Cross-Lingual Intervention
- 推理阶段通过最小二乘法学习跨语言对齐矩阵，实现表征空间转换。
- 在9个基准上，5个模型的多语言任务性能显著优于现有强基线。
- 零成本适配，适合急需多语支持但无训练资源的场景。
大型语言模型在自然语言处理中表现出色，但在不同语言间存在显著性能差距。现有方法多依赖预训练或微调，成本高昂。为克服这一局限，我们提出推理时跨语言干预（INCLINE）框架，通过在推理阶段将低性能语言的内部表征对齐至高性能语言空间，从而提升其表现。INCLINE首先利用源语言与目标语言的平行句对，通过最小二乘优化学习对齐矩阵，随后在推理过程中应用该矩阵，将低性能语言表征映射到高性能语言空间。在五个大模型和九个基准上的实验表明，相比近期强基线，INCLINE在多种任务和语言上均实现显著性能提升。分析显示，该方法成本极低且适用广泛。代码已开源：https://github.com/weixuan-wang123/INCLINE。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have shown remarkable capabilities in natural language processing but exhibit significant performance gaps among different languages. Most existing approaches to address these disparities rely on pretraining or fine-tuning, which are resource-intensive. To overcome these limitations without incurring significant costs, we propose Inference-Time Cross-Lingual Intervention (INCLINE), a novel framework that enhances LLM performance on low-performing (source) languages by aligning their internal representations with those of high-performing (target) languages during inference. INCLINE initially learns alignment matrices using parallel sentences from source and target languages through a Least-Squares optimization, and then applies these matrices during inference to transform the low-performing language representations toward the high-performing language space. Extensive experiments on nine benchmarks with five LLMs demonstrate that INCLINE significantly improves performance across diverse tasks and languages, compared to recent strong baselines. Our analysis demonstrates that INCLINE is highly cost-effective and applicable to a wide range of applications. In addition, we release the code to foster research along this line: https://github.com/weixuan-wang123/INCLINE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。