用古诗语义指导修复古画色彩,让画与诗重归和谐。
Reviving Ancient Paintings via Poem: A Colorization Framework for Aligning Cultural Semantics

- 通过古诗转配色预训练,将诗意转化为古典色彩条件。
- 引入结构感知注意力机制,精准控制古诗色彩注入位置和强度。
- 结合合成退化与专家修复数据,提升修复的历史真实性和文化契合度。
古代绘画因不可逆褪色,破坏了“诗画相合”这一核心美学原则,即视觉图像与文学题跋的和谐统一。尽管扩散模型具备强大的生成先验,但恢复历史真实色彩仍面临挑战:视觉修复本身具有模糊性,而直接文本引导常导致现代语义偏差、色彩过饱和及跨区域色溢。为此,我们提出PoemColor——一种诗文引导的古画着色框架。方法包含两项关键设计:首先,诗画投影器(P3)通过诗到配色的预训练,将隐含的诗意语境转化为具古典色彩意识的条件,降低诗到色映射的模糊性;其次,结构感知语义注意力(SASA)通过联合控制语义传播方向与区域注入强度,调节诗文色彩信息如何注入扩散主干网络。此外,我们构建了一个混合修复数据集,融合合成退化与专家修复成果,既提供可扩展监督信号,又保留真实的古典色彩参考。大量实验表明,该框架显著优于现有方法,实现可控着色,成功恢复历史真实感与诗画语义一致性。
原文摘要 · Abstract (English)
The irreversible fading of ancient paintings disrupts the "congruence between poems and paintings", a core aesthetic principle where visual imagery harmonizes with literary inscriptions. Although diffusion models provide strong generative priors, restoring historically faithful colors remains difficult: visual restoration is inherently ambiguous, while direct text guidance often causes modern semantic bias, over-saturation, and cross-boundary color leakage. To address this, we propose PoemColor, a poem-guided ancient painting colorization framework. Our method aligns poetic cultural semantics with painting restoration through two key designs. First, the Poetic Painting Projector (P3) converts implicit poetic context into a classical color-aware condition via poem-to-palette pretraining, reducing the ambiguity of poem-to-color mapping. Second, Structure-Aware Semantic Attention (SASA) regulates how poetic color semantics are injected into the diffusion backbone by jointly controlling their propagation direction and regional injection strength. In addition, we construct a hybrid restoration dataset that integrates synthetic degradation with expert-restored artifacts, providing both scalable supervision and real classical color references. Extensive experiments demonstrate that our framework significantly outperforms state-of-the-art methods, delivering controllable colorization that revives both historical authenticity and poetic semantics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。