用多尺度扩散模型修复敦煌壁画,细节还原更真实。
DiffuMural: Restoring Dunhuang Murals with Multi-scale Diffusion
- 融合多尺度与协同扩散机制,提升修复一致性。
- 在23幅敦煌壁画数据上实现高精度细节恢复。
- 兼顾美学与文化价值,适合文物保护研究者使用。
大规模预训练扩散模型在条件图像生成中表现优异,但古壁画修复因缺陷区域大、训练样本少,面临严峻挑战。现有方法缺乏对修复部分是否符合壁画整体风格与衔接细节的评估标准。为此,我们提出DiffuMural,结合多尺度收敛与协同扩散机制,引入ControlNet和循环一致性损失,优化生成图像与控制条件的匹配度。该模型基于23幅风格一致的敦煌壁画训练,能有效恢复精细细节,保持整体连贯性,并解决无真实依据的残缺壁画修复难题。评估框架包含四项量化指标:事实准确性、纹理细节、上下文语义与整体视觉一致性;同时融入人文价值评估,确保修复结果保留文化与艺术意义。大量实验表明,本方法在定性与定量指标上均优于现有最先进(SOTA)方法。
原文摘要 · Abstract (English)
Large-scale pre-trained diffusion models have produced excellent results in the field of conditional image generation. However, restoration of ancient murals, as an important downstream task in this field, poses significant challenges to diffusion model-based restoration methods due to its large defective area and scarce training samples. Conditional restoration tasks are more concerned with whether the restored part meets the aesthetic standards of mural restoration in terms of overall style and seam detail, and such metrics for evaluating heuristic image complements are lacking in current research. We therefore propose DiffuMural, a combined Multi-scale convergence and Collaborative Diffusion mechanism with ControlNet and cyclic consistency loss to optimise the matching between the generated images and the conditional control. DiffuMural demonstrates outstanding capabilities in mural restoration, leveraging training data from 23 large-scale Dunhuang murals that exhibit consistent visual aesthetics. The model excels in restoring intricate details, achieving a coherent overall appearance, and addressing the unique challenges posed by incomplete murals lacking factual grounding. Our evaluation framework incorporates four key metrics to quantitatively assess incomplete murals: factual accuracy, textural detail, contextual semantics, and holistic visual coherence. Furthermore, we integrate humanistic value assessments to ensure the restored murals retain their cultural and artistic significance. Extensive experiments validate that our method outperforms state-of-the-art (SOTA) approaches in both qualitative and quantitative metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。