无需标注数据,用语义信息自动修复过曝欠曝图像
CLIP-Guided Unsupervised Semantic-Aware Exposure Correction
- 用FastSAM提取语义,融合到图像特征中指导修正
- 通过CLIP生成伪标签,实现无监督训练
- 适合处理真实场景的过曝欠曝图像,尤其在细节恢复上表现好
不当曝光常导致细节丢失、色彩失真和对比度下降。曝光校正面临两大挑战:(1) 忽视物体级别的区域语义信息,引发色彩偏移伪影;(2) 现实世界曝光图像通常无真值标签,标注需大量人工编辑。为此,我们提出一种新的无监督语义感知曝光校正网络。该网络包含自适应语义感知融合模块,将预训练的FastSAM提取的语义信息有效融合至共享图像特征空间。融合特征由多尺度残差空间Mamba组用于细节恢复与曝光调整。为避免人工标注,我们设计了由CLIP引导的伪真值生成器,经微调后可自动识别曝光情况并指导针对性修正。同时,利用FastSAM和CLIP中的丰富先验,构建语义提示一致性损失,强化语义一致性与图像-提示对齐,实现无监督训练。大量实验表明,本方法在真实曝光图像校正上效果显著,优于当前最先进的无监督方法,在数值与视觉表现上均更优。
原文摘要 · Abstract (English)
Improper exposure often leads to severe loss of details, color distortion, and reduced contrast. Exposure correction still faces two critical challenges: (1) the ignorance of object-wise regional semantic information causes the color shift artifacts; (2) real-world exposure images generally have no ground-truth labels, and its labeling entails massive manual editing. To tackle the challenges, we propose a new unsupervised semantic-aware exposure correction network. It contains an adaptive semantic-aware fusion module, which effectively fuses the semantic information extracted from a pre-trained Fast Segment Anything Model into a shared image feature space. Then the fused features are used by our multi-scale residual spatial mamba group to restore the details and adjust the exposure. To avoid manual editing, we propose a pseudo-ground truth generator guided by CLIP, which is fine-tuned to automatically identify exposure situations and instruct the tailored corrections. Also, we leverage the rich priors from the FastSAM and CLIP to develop a semantic-prompt consistency loss to enforce semantic consistency and image-prompt alignment for unsupervised training. Comprehensive experimental results illustrate the effectiveness of our method in correcting real-world exposure images and outperforms state-of-the-art unsupervised methods both numerically and visually.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。