通过跨语言嵌入对齐，提升非英语模型性能
LANGALIGN: Enhancing Non-English Language Models via Cross-Lingual Embedding Alignment
- 在语言模型与任务头之间对齐英汉嵌入向量
- 韩、日、中文任务性能显著提升
- 支持反向转换，让非英语数据适配英语模型
尽管大语言模型受到关注，许多服务开发者仍因实际限制依赖基于嵌入的模型。此时，微调数据质量直接影响性能，而英语数据集常被用作训练非英语模型的初始数据。本文提出 LANGALIGN，通过在语言模型与任务头之间的接口对齐英语嵌入向量与目标语言嵌入向量，增强目标语言处理能力。在韩语、日语和汉语上的实验表明，LANGALIGN 在三种语言上均显著提升性能。此外，我们证明该方法可逆向使用，将目标语言数据转换为英语模型可处理的格式。
原文摘要 · Abstract (English)
While Large Language Models have gained attention, many service developers still rely on embedding-based models due to practical constraints. In such cases, the quality of fine-tuning data directly impacts performance, and English datasets are often used as seed data for training non-English models. In this study, we propose LANGALIGN, which enhances target language processing by aligning English embedding vectors with those of the target language at the interface between the language model and the task header. Experiments on Korean, Japanese, and Chinese demonstrate that LANGALIGN significantly improves performance across all three languages. Additionally, we show that LANGALIGN can be applied in reverse to convert target language data into a format that an English-based model can process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。