提升红外图像生成细节，让小目标检测更准。
Addressing Detail Bottlenecks in Latent Diffusion for RGB-to-SWIR Image Translation
- 用高分辨率源图特征跳连注入解码器，保留空间细节
- 小目标检测准确率最高提升3.4倍，整体检测性能翻倍
- 适合做遥感、自动驾驶中细粒度图像转换的研究者
潜在扩散模型（LDM）虽能高效实现图像到图像的转换，但在压缩过程中会丢失精细的空间细节，影响下游感知任务。我们识别出两个瓶颈：自动编码器在压缩中损失空间信息，条件路径通过简单下采样进一步劣化源信号。为此提出两种轻量级、与主干无关的改进：源条件自动编码器（SCAE），通过跳跃连接将高分辨率源特征注入解码器；可学习引导编码器（LGE），以学习到的条件信号替代原始下采样。在驾驶场景的RGB到短波红外（SWIR）图像转换任务上，使用两种去噪器主干（U-Net和DiT），本方法使检测平均精度（mAP）相比基线最高提升2倍，对小物体（COCO-small，面积<32²像素²）提升达3.4倍，同时达到当前最优的FID分数。进一步发现FID与检测性能相关性差，支持多维度评估。结果在公开的RASMD基准上实现零样本泛化。所有测试数据、标注、检查点及训练代码将公开发布。
原文摘要 · Abstract (English)
Latent diffusion models (LDMs) enable efficient image-to-image translation but discard fine spatial details during compression, degrading downstream perception tasks. We identify two bottlenecks: the autoencoder, which loses spatial information, and the conditioning pathway, which further degrades the source signal through naive downsampling. We propose two lightweight, backbone-agnostic fixes: a Source-Conditioned Autoencoder (SCAE) that injects high-resolution source features into the decoder via skip connections, and a Learnable Guidance Encoder (LGE) that replaces naive downsampling with a learned conditioning signal. Evaluated on RGB-to-SWIR translation for driving scenes with two denoiser backbones (U-Net and DiT), our approach improves detection mAP by up to 2x over the latent diffusion baseline, with up to 3.4x gains on small objects (COCO-small, <32^2 px^2), while achieving state-of-the-art FID. We further show that FID and detection performance are poorly correlated, motivating multi-axis evaluation. Results generalise zero-shot to the public RASMD benchmark. We will publicly release test data with annotations, all checkpoints, and training code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。