用OCR与AI自动解析16-19世纪大学教授档案，实现历史数据精准入库。
Enriching Historical Records: An OCR and AI-Driven Approach for Database Integration
- 结合OCR与生成式AI，结构化提取手写体历史文献信息
- OCR字符错误率仅1.08%，数据链接准确率达81%-94%
- 适合数字人文、历史档案数字化研究者参考
本研究对1983至1985年间编纂的《莱顿大学教授与讲师记录（1575–1815）》进行数字化处理，探讨如何构建自动化流程，将历史文档图像中的文本通过OCR、大语言模型辅助解析与数据库关联，整合进高质量数据库。采用OCR技术对打字体历史文献进行识别，获得1.08%的字符错误率（CER）和5.06%的词错误率（WER）；基于标注数据的JSON结构化提取平均准确率达65%，未标注情况下为63%。生成式AI在一定程度上弥补了低质量OCR表现。记录链接算法对标注后的JSON文件匹配准确率达94%，对直接由OCR生成的JSON文件匹配准确率为81%。该研究为数字人文领域提供了可复用的自动化管道，解决了版面差异与术语不一致等挑战，验证了先进生成式AI在历史文献处理中的有效性。
原文摘要 · Abstract (English)
This research digitizes and analyzes the Leidse hoogleraren en lectoren 1575-1815 books written between 1983 and 1985, which contain biographic data about professors and curators of Leiden University. It addresses the central question: how can we design an automated pipeline that integrates OCR, LLM-based interpretation, and database linking to harmonize data from historical document images with existing high-quality database records? We applied OCR techniques, generative AI decoding constraints that structure data extraction, and database linkage methods to process typewritten historical records into a digital format. OCR achieved a Character Error Rate (CER) of 1.08 percent and a Word Error Rate (WER) of 5.06 percent, while JSON extraction from OCR text achieved an average accuracy of 63 percent and, based on annotated OCR, 65 percent. This indicates that generative AI somewhat corrects low OCR performance. Our record linkage algorithm linked annotated JSON files with 94% accuracy and OCR-derived JSON files with 81%. This study contributes to digital humanities research by offering an automated pipeline for interpreting digitized historical documents, addressing challenges like layout variability and terminology differences, and exploring the applicability and strength of an advanced generative AI model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。