通过测试时图像增强提升历史文档文字提取准确率
Improving MLLM Historical Record Extraction with Test-Time Image
- 对同一图像生成多个增强版本，用自定义对齐算法融合输出
- 在622份宾夕法尼亚死亡记录上准确率提升4个百分点
- 适合需要高精度历史文献数字化的项目团队
我们提出一种新型集成框架，稳定基于大语言模型的历史文档文字提取。对每张图像生成多个增强变体，使用Gemini 2.0 Flash进行转录，并通过自定义的Needleman-Wunsch风格对齐器融合输出，得到共识转录结果和置信度评分。我们构建了一个包含622份宾夕法尼亚州死亡记录的新数据集，实验表明该方法相比单次输入基线准确率提升4个百分点。研究发现，填充和模糊对提升准确率最有效，而网格扭曲扰动最有助于区分高低置信度案例。该方法简单、可扩展，可立即部署于其他文档集合与转录模型。
原文摘要 · Abstract (English)
We present a novel ensemble framework that stabilizes LLM based text extraction from noisy historical documents. We transcribe multiple augmented variants of each image with Gemini 2.0 Flash and fuse these outputs with a custom Needleman Wunsch style aligner that yields both a consensus transcription and a confidence score. We present a new dataset of 622 Pennsylvania death records, and demonstrate our method improves transcription accuracy by 4 percentage points relative to a single shot baseline. We find that padding and blurring are the most useful for improving accuracy, while grid warp perturbations are best for separating high and low confidence cases. The approach is simple, scalable, and immediately deployable to other document collections and transcription models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。