用像素编码突破词汇表限制，提升多语言模型表现
Overcoming Vocabulary Constraints with Pixel-level Fallback
- 用文本渲染成像素的方式生成嵌入，绕过传统词汇表
- 在机器翻译上超越基于分词和字节的方法，跨语言迁移更有效
- 适合希望低成本增强多语言能力的开发者
子词分词需在计算效率与词汇覆盖间权衡，常导致非主流语言和文字表现不佳。我们提出在预训练语言模型中引入无词汇表编码器，通过将文本渲染为像素生成输入嵌入。在以英语为中心的模型上实验表明，该方法显著提升机器翻译性能，促进有效跨语言迁移，优于基于分词的方法。此外，像素表示优于字节级方法和标准词汇扩展。该方法在无需大量重训练的情况下增强单语模型的多语言能力，并通过输入压缩降低解码延迟。
原文摘要 · Abstract (English)
Subword tokenization requires balancing computational efficiency and vocabulary coverage, which often leads to suboptimal performance on languages and scripts not prioritized during training. We propose to augment pretrained language models with a vocabulary-free encoder that generates input embeddings from text rendered as pixels. Through experiments on English-centric language models, we demonstrate that our approach substantially improves machine translation performance and facilitates effective cross-lingual transfer, outperforming tokenizer-based methods. Furthermore, we find that pixel-based representations outperform byte-level approaches and standard vocabulary expansion. Our approach enhances the multilingual capabilities of monolingual language models without extensive retraining and reduces decoding latency via input compression.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。