用SDR图像语义知识提升低质量HDR重建效果
Boosting HDR Image Reconstruction via Semantic Knowledge Transfer
- 从SDR图像提取语义先验,指导HDR重建
- 自蒸馏机制对齐基线与改进模型的输出
- 适合需要提升现有HDR方法性能的研究者
从多张存在明显退化和内容缺失的SDR图像中恢复高动态范围(HDR)图像极具挑战。利用场景特定的语义先验可有效修复严重退化的区域。然而,这些先验通常来自sRGB SDR图像,其与HDR成像之间存在显著域差异。为此,我们提出一个通用框架,通过自蒸馏将SDR域的语义知识迁移至HDR重建中以提升性能。首先,设计语义先验引导重建模型(SPGRM),利用SDR语义知识解决初始HDR重建中的病态问题;其次,采用自蒸馏机制,约束颜色与内容信息以对齐基线模型与SPGRM的外部输出;此外,引入语义知识对齐模块(SKAM),通过互补掩码填补内部特征中的缺失语义内容。大量实验表明,该框架在不修改网络结构的前提下,显著提升了现有方法的HDR重建质量。
原文摘要 · Abstract (English)
Recovering High Dynamic Range (HDR) images from multiple Standard Dynamic Range (SDR) images become challenging when the SDR images exhibit noticeable degradation and missing content. Leveraging scene-specific semantic priors offers a promising solution for restoring heavily degraded regions. However, these priors are typically extracted from sRGB SDR images, the domain/format gap poses a significant challenge when applying it to HDR imaging. To address this issue, we propose a general framework that transfers semantic knowledge derived from SDR domain via self-distillation to boost existing HDR reconstruction. Specifically, the proposed framework first introduces the Semantic Priors Guided Reconstruction Model (SPGRM), which leverages SDR image semantic knowledge to address ill-posed problems in the initial HDR reconstruction results. Subsequently, we leverage a self-distillation mechanism that constrains the color and content information with semantic knowledge, aligning the external outputs between the baseline and SPGRM. Furthermore, to transfer the semantic knowledge of the internal features, we utilize a Semantic Knowledge Alignment Module (SKAM) to fill the missing semantic contents with the complementary masks. Extensive experiments demonstrate that our framework significantly boosts HDR imaging quality for existing methods without altering the network architecture.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。