低资源语言间可互学,用共享知识提升识别效果。
What Can Low Resource Languages Learn From Each Other?

- 构建多语言共享模型,融合不同语言专家优势。
- 在10种印度文字上实现平均2%的识别率提升。
- 适合资源匮乏语言的OCR系统开发与研究者。
尽管视觉-语言模型(VLMs)发展迅速,其语言覆盖仍主要局限于高资源语言,导致全球7000多种语言中的多数被排除在数字鸿沟之外。这一差距在光学字符识别(OCR)中尤为显著,低资源文字缺乏大规模数据以支持传统扩展规律。本文研究极端数据稀缺场景下的OCR适应问题(真实图像少于10,000张,合成图像少于250,000张),发现常规微调策略易达性能瓶颈。关键发现显示:专用模型的高层虽分化以捕捉特定文字特征,但底层却学习冗余、高度相似的特征。为此,我们提出PSMC框架(预训练、专业化、合并、协同训练),利用跨语言“迁移效应”。该方法先从高资源基模型生成语言专属专家,再通过任务算术将它们融合为统一的高性能多语言骨干。在10种印度文字(支持20+语言)上的实验表明,PSMC在不增加参数量的前提下,平均提升2%的字词识别率(WRR)。结果表明,融合后的潜在空间协同训练能促进有益的知识迁移,为包容性VLM的发展提供可扩展路径。源代码与数据集将在发表后公开。
原文摘要 · Abstract (English)
Despite the rapid advancement of Vision-Language Models (VLMs), their linguistic reach remains largely confined to high-resource languages, leaving the majority of the world's 7,000+ living languages on the wrong side of a growing digital divide. This disparity is especially pronounced in Optical Character Recognition (OCR), where low-resource scripts lack the massive datasets required for traditional scaling laws. We investigate OCR adaptation in extreme data-scarce regimes (<10K real and <250K synthetic images), demonstrating that conventional fine-tuning strategies often reach a performance ceiling. Our key finding reveals a structural inefficiency in language-specific adaptation: while higher layers of specialized models diverge to capture unique script nuances, the lower layers learn redundant, highly similar features. Motivated by this observation, we propose PSMC (Pre-train, Specialize, Merge, and Co-train), a data-efficient framework that capitalizes on a cross-script "transfer effect". Our approach first derives language-specific experts from a high-resource base model, then employs task arithmetic to fuse these experts into a unified, high-performance multilingual back- bone. Extensive evaluation across 10 Indian scripts (supporting 20+ languages) shows that PSMC achieves a ~2% average improvement in Word Recognition Rate (WRR) over individual specialist models without increasing parameter count. Our results indicate that joint training in the merged latent space facilitates a constructive knowledge transfer that benefits all constituent scripts, providing a scalable pathway for inclusive VLM development. Source code and datasets will be released post publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。