用流模型生成热成像,支持多视角多环境真实合成。
ThermalGen: Style-Disentangled Flow-Based Generative Models for RGB-to-Thermal Image Translation
- 基于流的生成模型,分离图像风格与内容
- 在8个数据集上超越现有生成方法
- 首个能处理视角、传感器、环境差异的热成像生成模型
成对的RGB-热成像数据对视觉-热传感器融合及跨模态任务至关重要,但同步校准的成对数据稀缺。为此,本文提出ThermalGen,一种用于RGB到热成像翻译的自适应流生成模型,结合图像条件化架构和风格解耦机制。为支持大规模训练,我们整理了8个公开的卫星-航拍、航拍及地面级成对数据集,并新增三个大规模卫星-航拍级数据集:DJI-day、Bosonplus-day 和 Bosonplus-night,覆盖不同时间、传感器类型和地理区域。在多个基准测试上的广泛评估表明,ThermalGen在翻译性能上达到或优于现有的基于GAN和扩散模型的方法。据我们所知,ThermalGen是首个能够生成反映显著视角、传感器特性及环境条件变化的热成像的模型。
原文摘要 · Abstract (English)
Paired RGB-thermal data is crucial for visual-thermal sensor fusion and cross-modality tasks, including important applications such as multi-modal image alignment and retrieval. However, the scarcity of synchronized and calibrated RGB-thermal image pairs presents a major obstacle to progress in these areas. To overcome this challenge, RGB-to-Thermal (RGB-T) image translation has emerged as a promising solution, enabling the synthesis of thermal images from abundant RGB datasets for training purposes. In this study, we propose ThermalGen, an adaptive flow-based generative model for RGB-T image translation, incorporating an RGB image conditioning architecture and a style-disentangled mechanism. To support large-scale training, we curated eight public satellite-aerial, aerial, and ground RGB-T paired datasets, and introduced three new large-scale satellite-aerial RGB-T datasets--DJI-day, Bosonplus-day, and Bosonplus-night--captured across diverse times, sensor types, and geographic regions. Extensive evaluations across multiple RGB-T benchmarks demonstrate that ThermalGen achieves comparable or superior translation performance compared to existing GAN-based and diffusion-based methods. To our knowledge, ThermalGen is the first RGB-T image translation model capable of synthesizing thermal images that reflect significant variations in viewpoints, sensor characteristics, and environmental conditions. Project page: http://xjh19971.github.io/ThermalGen
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。