对比扩散与流模型在夜间灯光数据融合中的表现,发现UNet结构扩散模型更保细节。
Exploring the design space of diffusion and flow models for data fusion
- 用UNet、扩散与流模型融合DMSP-OLS和VIIRS夜间灯光数据
- 扩散模型生成图像保真度高,能更好保留空间细节
- 推荐用离散噪声调度提升质量,量化可降开销不损性能
数据融合在多个领域至关重要,尤其在卫星遥感中可通过融合多源观测提升时空分辨率。本研究探索扩散与流模型在数据融合中的设计空间,聚焦于融合防御气象卫星计划运行线扫描系统(DMSP-OLS)与可见红外成像辐射计套件(VIIRS)的夜间灯光数据。采用多种2D图像到图像生成模型,包括UNet、扩散模型和流模型架构。结果表明,基于UNet的扩散模型在保持精细空间细节和生成高质量融合图像方面表现尤为出色。同时,研究分析了噪声调度策略:迭代求解器加速推理,离散调度器则获得更高重建质量。此外,探索了量化技术,在不损失性能的前提下降低内存占用与计算成本。研究为遥感数据融合任务中选择最优扩散与流模型架构提供了实用指导,并建议通过合理噪声调度提升融合效果。
原文摘要 · Abstract (English)
Data fusion is an essential task in various domains, enabling the integration of multi-source information to enhance data quality and insights. One key application is in satellite remote sensing, where fusing multi-sensor observations can improve spatial and temporal resolution. In this study, we explore the design space of diffusion and flow models for data fusion, focusing on the integration of Defense Meteorological Satellite Program's Operational Linescan System (DMSP-OLS) and Visible Infrared Imaging Radiometer Suite (VIIRS) nighttime lights data. Our approach leverages a diverse set of 2D image-to-image generative models, including UNET, diffusion, and flow modeling architectures. We evaluate the effectiveness of these architectures in satellite remote sensing data fusion, identifying diffusion models based on UNet as particularly adept at preserving fine-grained spatial details and generating high-fidelity fused images. We also provide guidance on the selection of noise schedulers in diffusion-based models, highlighting the trade-offs between iterative solvers for faster inference and discrete schedulers for higher-quality reconstructions. Additionally, we explore quantization techniques to optimize memory efficiency and computational cost without compromising performance. Our findings offer practical insights into selecting the most effective diffusion and flow model architectures for data fusion tasks, particularly in remote sensing applications, and provide recommendations for leveraging noise scheduling strategies to enhance fusion quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。