让大模型在资源少的语言上表现更好，通过翻译+适配器提升性能
TALL -- A Trainable Architecture for Enhancing LLM Performance in Low-Resource Languages
- 用双语翻译模型将低资源语言转为高资源语言表征
- 在希伯来语上效果显著优于直接使用、简单翻译等方法
- 只训练轻量适配模块，保持高效且不破坏预训练模型
大型语言模型在高资源语言上表现优异，但在低资源语言上因训练数据有限而表现不佳。本文提出TALL（可训练架构，用于增强低资源语言下的大模型性能），将大模型与两个双语翻译模型结合，将低资源输入转化为高资源表征，利用大模型能力的同时，通过维度对齐层和自定义变换器保留语言特征。在希伯来语上的实验显示，TALL显著优于多个基线方法，包括直接使用、简单翻译和微调方案。该架构采用参数高效策略，冻结预训练组件，仅训练轻量适配模块，在保持计算效率的同时实现性能提升。
原文摘要 · Abstract (English)
Large Language Models (LLMs) excel in high-resource languages but struggle with low-resource languages due to limited training data. This paper presents TALL (Trainable Architecture for Enhancing LLM Performance in Low-Resource Languages), which integrates an LLM with two bilingual translation models. TALL transforms low-resource inputs into high-resource representations, leveraging the LLM's capabilities while preserving linguistic features through dimension alignment layers and custom transformers. Our experiments on Hebrew demonstrate significant improvements over several baselines, including direct use, naive translation, and fine-tuning approaches. The architecture employs a parameter-efficient strategy, freezing pre-trained components while training only lightweight adapter modules, balancing computational efficiency with performance gains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。