arXiv:2608.27753cs.CV2026-08

低资源语言间可互学,用共享知识提升识别效果。

What Can Low Resource Languages Learn From Each Other?

论文配图:What Can Low Resource Languages Learn From Each Other?
图 1 · 摘自论文原文
  • 构建多语言共享模型,融合不同语言专家优势。
  • 在10种印度文字上实现平均2%的识别率提升。
  • 适合资源匮乏语言的OCR系统开发与研究者。

尽管视觉-语言模型(VLMs)发展迅速,其语言覆盖仍主要局限于高资源语言,导致全球7000多种语言中的多数被排除在数字鸿沟之外。这一差距在光学字符识别(OCR)中尤为显著,低资源文字缺乏大规模数据以支持传统扩展规律。本文研究极端数据稀缺场景下的OCR适应问题(真实图像少于10,000张,合成图像少于250,000张),发现常规微调策略易达性能瓶颈。关键发现显示:专用模型的高层虽分化以捕捉特定文字特征,但底层却学习冗余、高度相似的特征。为此,我们提出PSMC框架(预训练、专业化、合并、协同训练),利用跨语言“迁移效应”。该方法先从高资源基模型生成语言专属专家,再通过任务算术将它们融合为统一的高性能多语言骨干。在10种印度文字(支持20+语言)上的实验表明,PSMC在不增加参数量的前提下,平均提升2%的字词识别率(WRR)。结果表明,融合后的潜在空间协同训练能促进有益的知识迁移,为包容性VLM的发展提供可扩展路径。源代码与数据集将在发表后公开。

原文摘要 · Abstract (English)

Despite the rapid advancement of Vision-Language Models (VLMs), their linguistic reach remains largely confined to high-resource languages, leaving the majority of the world's 7,000+ living languages on the wrong side of a growing digital divide. This disparity is especially pronounced in Optical Character Recognition (OCR), where low-resource scripts lack the massive datasets required for traditional scaling laws. We investigate OCR adaptation in extreme data-scarce regimes (<10K real and <250K synthetic images), demonstrating that conventional fine-tuning strategies often reach a performance ceiling. Our key finding reveals a structural inefficiency in language-specific adaptation: while higher layers of specialized models diverge to capture unique script nuances, the lower layers learn redundant, highly similar features. Motivated by this observation, we propose PSMC (Pre-train, Specialize, Merge, and Co-train), a data-efficient framework that capitalizes on a cross-script "transfer effect". Our approach first derives language-specific experts from a high-resource base model, then employs task arithmetic to fuse these experts into a unified, high-performance multilingual back- bone. Extensive evaluation across 10 Indian scripts (supporting 20+ languages) shows that PSMC achieves a ~2% average improvement in Word Recognition Rate (WRR) over individual specialist models without increasing parameter count. Our results indicate that joint training in the merged latent space facilitates a constructive knowledge transfer that benefits all constituent scripts, providing a scalable pathway for inclusive VLM development. Source code and datasets will be released post publication.

OCR多语言低资源迁移学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。