让低资源语言像图像一样被模型理解，提升跨语言能力
Languages are Modalities: Cross-Lingual Alignment via Encoder Injection
- 用轻量投影将多语编码器嵌入注入解码器潜空间
- 在问答评估中比基线高63.6%，检索准确率达81.3%
- 无需改分词器或重训，适合资源少的语言
指令微调的大语言模型在低资源非拉丁语系上表现不佳，主要因分词碎片化与跨语言耦合弱。我们提出LLINK（非英语知识的潜在语言注入），一种计算高效的语言作为模态方法，通过不改变分词器或重训解码器来实现条件化。首先，利用轻量对比投影器，将冻结的多语言编码器句向量对齐至解码器潜空间的预留位置；其次，该向量扩展为K个软槽，并通过最小适配器训练，使冻结解码器接收信号。LLINK显著提升双语检索性能，在大模型评判的问答评估中，相比基线模型高出81.3%，优于直接微调63.6%。进一步发现，性能提升源于减少分词膨胀和增强跨语言对齐，尽管模型在数值保真性上仍存弱点。将低资源语言视为模态，为轻量级大模型提供更强跨语言对齐的实用路径。
原文摘要 · Abstract (English)
Instruction-tuned Large Language Models (LLMs) underperform on low resource, non-Latin scripts due to tokenizer fragmentation and weak cross-lingual coupling. We present LLINK (Latent Language Injection for Non-English Knowledge), a compute efficient language-as-modality method that conditions an instruction-tuned decoder without changing the tokenizer or retraining the decoder. First, we align sentence embeddings from a frozen multilingual encoder to the decoder's latent embedding space at a reserved position via a lightweight contrastive projector. Second, the vector is expanded into K soft slots and trained with minimal adapters so the frozen decoder consumes the signal. LLINK substantially improves bilingual retrieval and achieves 81.3% preference over the base model and 63.6% over direct fine-tuning in LLM-judged Q&A evaluations. We further find that improvements can be attributed to reduced tokenization inflation and a stronger cross lingual alignment, despite the model having residual weaknesses in numeric fidelity. Treating low resource languages as a modality offers a practical path to stronger cross-lingual alignment in lightweight LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。