arXiv:2412.11634cs.CV2024-12AAAI被引 13

用扩散模型修复破损古籍,还原字迹原貌。

Predicting the Original Appearance of Damaged Historical Documents

  • 基于扩散模型与字符感知损失,融合语义与空间信息
  • 在28,552对图像上训练,显著优于现有方法
  • 适用于古籍修复、文本块生成,适合文化遗产研究

历史文献蕴含丰富文化宝藏,但长期受字迹缺失、纸张损毁、墨迹褪色等影响。现有文档处理方法多聚焦二值化、增强,忽视损伤修复。为此,我们提出新任务——历史文献修复(HDR),旨在预测破损古籍的原始外观。为填补空白,我们构建了大规模数据集HDR28K,包含28,552对破损-修复图像,带字符级标注和多风格退化。同时提出基于扩散的网络DiffHDR,通过引入语义与空间信息,结合精心设计的字符感知损失,实现上下文与视觉一致性。实验表明,使用HDR28K训练的DiffHDR显著超越现有方法,在真实破损文档修复中表现优异。此外,DiffHDR可拓展至文档编辑与文本块生成,展现高灵活性与泛化能力。本研究或开启文档处理新方向,助力珍贵文化传承。数据集与代码见https://github.com/yeungchenwa/HDR。

原文摘要 · Abstract (English)

Historical documents encompass a wealth of cultural treasures but suffer from severe damages including character missing, paper damage, and ink erosion over time. However, existing document processing methods primarily focus on binarization, enhancement, etc., neglecting the repair of these damages. To this end, we present a new task, termed Historical Document Repair (HDR), which aims to predict the original appearance of damaged historical documents. To fill the gap in this field, we propose a large-scale dataset HDR28K and a diffusion-based network DiffHDR for historical document repair. Specifically, HDR28K contains 28,552 damaged-repaired image pairs with character-level annotations and multi-style degradations. Moreover, DiffHDR augments the vanilla diffusion framework with semantic and spatial information and a meticulously designed character perceptual loss for contextual and visual coherence. Experimental results demonstrate that the proposed DiffHDR trained using HDR28K significantly surpasses existing approaches and exhibits remarkable performance in handling real damaged documents. Notably, DiffHDR can also be extended to document editing and text block generation, showcasing its high flexibility and generalization capacity. We believe this study could pioneer a new direction of document processing and contribute to the inheritance of invaluable cultures and civilizations. The dataset and code is available at https://github.com/yeungchenwa/HDR.

古籍修复扩散模型图像修复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。