融合物理模型的3D高斯方法，提升热红外新视角合成清晰度。
Thermal3D-GS: Physics-induced 3D Gaussians for Thermal Infrared Novel-view Synthesis
- 用神经网络建模大气透射与热传导，驱动3D高斯渲染。
- 在自建数据集上实现PSNR提升3.03 dB，消除伪影和模糊边界。
- 适合做热成像重建、夜间/恶劣天气视觉任务的研究者。
基于可见光的新视角合成已广泛研究，而热红外成像具备全天候成像和强穿透能力，有助于夜间及恶劣天气下的场景重建。然而，热红外受大气透射效应和热传导等物理特性影响，导致合成图像中出现漂浮物和边缘模糊等问题。为此，本文提出物理诱导的3D高斯泼溅方法Thermal3D-GS，通过神经网络建模三维介质中的大气透射与热传导，并引入温度一致性约束优化目标以提升重建精度。此外，构建了首个大规模热红外新视角合成基准数据集TI-NSD，包含20个真实热红外视频场景（涵盖室内、室外与无人机场景），共6,664帧热红外图像。实验表明，该方法相比基线在PSNR上提升3.03 dB，显著改善漂浮物与模糊边界问题。代码与数据集将公开于GitHub。
原文摘要 · Abstract (English)
Novel-view synthesis based on visible light has been extensively studied. In comparison to visible light imaging, thermal infrared imaging offers the advantage of all-weather imaging and strong penetration, providing increased possibilities for reconstruction in nighttime and adverse weather scenarios. However, thermal infrared imaging is influenced by physical characteristics such as atmospheric transmission effects and thermal conduction, hindering the precise reconstruction of intricate details in thermal infrared scenes, manifesting as issues of floaters and indistinct edge features in synthesized images. To address these limitations, this paper introduces a physics-induced 3D Gaussian splatting method named Thermal3D-GS. Thermal3D-GS begins by modeling atmospheric transmission effects and thermal conduction in three-dimensional media using neural networks. Additionally, a temperature consistency constraint is incorporated into the optimization objective to enhance the reconstruction accuracy of thermal infrared images. Furthermore, to validate the effectiveness of our method, the first large-scale benchmark dataset for this field named Thermal Infrared Novel-view Synthesis Dataset (TI-NSD) is created. This dataset comprises 20 authentic thermal infrared video scenes, covering indoor, outdoor, and UAV(Unmanned Aerial Vehicle) scenarios, totaling 6,664 frames of thermal infrared image data. Based on this dataset, this paper experimentally verifies the effectiveness of Thermal3D-GS. The results indicate that our method outperforms the baseline method with a 3.03 dB improvement in PSNR and significantly addresses the issues of floaters and indistinct edge features present in the baseline method. Our dataset and codebase will be released in \href{https://github.com/mzzcdf/Thermal3DGS}{\textcolor{red}{Thermal3DGS}}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。