用合成数据微调视觉语言模型，实现满语手写文档高精度识别
Finetuning Vision-Language Models as OCR Systems for Low-Resource Languages: A Case Study of Manchu
- 通过参数高效微调三个视觉语言模型，在合成数据上训练满语文字识别系统
- 模型在合成数据上达98.3%词准确率，真实手写文档仍保持93.1%准确率
- 为濒危语言数字保护提供低成本可复用方案，适合历史与语言学者使用
满语是理解早期现代欧亚东部历史的关键濒危语言，但缺乏能处理真实历史文献的光学字符识别（OCR）系统。本研究通过在6万张合成满语单词图像上微调三个开源视觉语言模型（LLaMA-3.2-11B、Qwen2.5-VL-7B、Qwen2.5-VL-3B），构建高性能OCR系统，采用参数高效训练方法。LLaMA-3.2-11B在合成数据上取得98.3%词准确率和0.0024字符错误率，关键是在真实手写文档上仍保持93.1%准确率。对比实验表明，传统CRNN基线在合成数据上达99.8%准确率，但在真实文档上骤降至72.5%。本方法展示出出色的从合成到真实场景的域迁移能力，提供可在普通硬件部署的低成本解决方案。该工作建立了一套可迁移的濒危语言OCR框架，降低数字人文领域的技术与经济门槛，使历史学家与语言学家无需专用计算资源即可处理历史档案。代码与模型权重见：https://github.com/mic7ch1/ManchuAI-OCR。
原文摘要 · Abstract (English)
Manchu, a critically endangered language essential for understanding early modern Eastern Eurasian history, lacks effective OCR systems that can handle real-world historical documents. This study develops high-performing OCR systems by fine-tuning three open-source vision-language models (LLaMA-3.2-11B, Qwen2.5-VL-7B, Qwen2.5-VL-3B) on 60,000 synthetic Manchu word images using parameter-efficient training. LLaMA-3.2-11B achieved exceptional performance with 98.3\% word accuracy and 0.0024 character error rate on synthetic data, while crucially maintaining 93.1\% accuracy on real-world handwritten documents. Comparative evaluation reveals substantial advantages over traditional approaches: while a CRNN baseline achieved 99.8\% synthetic accuracy, it suffered severe degradation to 72.5\% on real documents. Our approach demonstrates effective synthetic-to-real domain transfer, providing a cost-effective solution deployable on accessible infrastructure. This work establishes a transferable framework for endangered language OCR that removes technical and financial barriers in digital humanities, enabling historians and linguists to process historical archives without specialized computing resources. Code and model weights are available at https://github.com/mic7ch1/ManchuAI-OCR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。