用RNN还原中世纪手稿的字符简化与缩写，提升文字识别准确率。
Letter Lemmatization: One-to-one and Banded RNNs for Reversing Character-Set Simplification and Abbreviation in Medieval Text
- 设计一对一RNN模型，通过自监督学习还原字符集变换。
- 在仅20行文本下恢复了50%的字符错误率（CER），显著提升识别精度。
- 提出字符语义相似性度量，适用于不同字符集间的映射转换，适合历史文献处理者。
中世纪文献转录实践多样，数字化策略不一，导致字符集呈现流动性。本文解决在灵活模式下字符集转换的问题，聚焦一对一字符映射，训练字符级一对一RNN以自监督方式逆向还原；即使仅有20个文本行，仍可恢复一半的字符错误率（CER）。分析该方法在光学字符识别后处理中的应用，发现其在忽略插入/删除的情况下实现显著性能提升。随后，利用从平行语料库构建的字符级对齐真值，采用名为‘带状RNN’（Banded RNNs）的训练与推理模式，成功扩展中世纪宪章转录中的缩写。最后，提出一种精细启发式方法，基于任意两字符集定义语义相似性度量，称之为‘字母词源化’（letter lemmatization），并发布一个高效的Python工具库，支持所有方法的实现。
原文摘要 · Abstract (English)
Medieval document transcribers have very different practices; on top of that, heterogeneous digitization policies have resulted in corpora where the character-set must be viewed as fluid. In this paper we address the problem of changing between character-sets in a flexible manner. We focus on one-to-one character mappings and train characterlevel one-to-one RNNs to undo them with self-supervision; recovering half the CER even with 20 text lines. We analyse the use of these one-to-one networks for HTR post-correction and we see that they obtain significant improvements while totally ignoring ins-dels. We then use the exact same networks with character-level alignment groundtruth compiled from parallel corpora in a training and inference mode we call Banded RNNs. We use such networks to successfully expand abbreviations in medieval charter transcriptions. Finally we introduce an elaborate heuristic which takes the characters of two arbitrary character-sets and defines a metric encapsulating what we consider to be semantic similarity of characters. We call the construction of such mappings letter lemmatization and present a rich Python library that efficiently performs all presented methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。