无需配对图像,用CLIP优化实现矿井弱光下图像实时增强。
CLIP-Optimized Multimodal Image Enhancement via ISP-CNN Fusion for Coal Mine IoVT under Uneven Illumination
- 融合ISP与CNN的两阶段架构,兼顾全局与细节增强。
- 在弱光场景下,PSNR提升2.9%-4.9%,SSIM增4.3%-11.4%。
- 适合边缘设备部署,适用于真实矿井监控系统。
清晰的监控图像对煤矿物联网(IoVT)系统的安全运行至关重要。然而,地下环境光照不足且亮度不均严重降低图像质量,而现有增强方法常依赖难以获取的成对参考图像。此外,边缘设备上的性能与效率之间存在权衡。为此,我们提出一种面向煤矿IoVT的多模态图像增强方法,采用基于CLIP的多模态迭代优化策略,实现无监督训练,并结合ISP-CNN融合架构,在两阶段中分别完成全局增强与细节优化,显著改善暗区图像质量。该方法通过集成传统图像信号处理(ISP)与卷积神经网络(CNN),在保持高性能的同时降低计算复杂度,适用于边缘设备实时部署。实验表明,相比七种先进算法,本方法在模拟矿井场景中使PSNR提升2.9%-4.9%,SSIM提升4.3%-11.4%,VIF提升4.9%-17.8%。结果验证了其在性能与计算开销间的良好平衡,支持更安全的采矿作业。
原文摘要 · Abstract (English)
Clear monitoring images are crucial for the safe operation of coal mine Internet of Video Things (IoVT) systems. However, low illumination and uneven brightness in underground environments significantly degrade image quality, posing challenges for enhancement methods that often rely on difficult-to-obtain paired reference images. Additionally, there is a trade-off between enhancement performance and computational efficiency on edge devices within IoVT systems.To address these issues, we propose a multimodal image enhancement method tailored for coal mine IoVT, utilizing an ISP-CNN fusion architecture optimized for uneven illumination. This two-stage strategy combines global enhancement with detail optimization, effectively improving image quality, especially in poorly lit areas. A CLIP-based multimodal iterative optimization allows for unsupervised training of the enhancement algorithm. By integrating traditional image signal processing (ISP) with convolutional neural networks (CNN), our approach reduces computational complexity while maintaining high performance, making it suitable for real-time deployment on edge devices.Experimental results demonstrate that our method effectively mitigates uneven brightness and enhances key image quality metrics, with PSNR improvements of 2.9%-4.9%, SSIM by 4.3%-11.4%, and VIF by 4.9%-17.8% compared to seven state-of-the-art algorithms. Simulated coal mine monitoring scenarios validate our method's ability to balance performance and computational demands, facilitating real-time enhancement and supporting safer mining operations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。