仅用热成像与深度估计实现高效3D场景重建，提升渲染质量并大幅缩短训练时间。
Supercharging Thermal Gaussian Splatting with Depth Estimation
- 基于热成像和深度估计构建单模态高斯点云，避免多模态融合复杂性。
- 在两个数据集上平均提升LPIPS、SSIM、PSNR指标，分别高出1.12%、0.034%、0.01%。
- 训练时间减少55%（12分47秒），适合实时应用如搜救、工业检测等场景。
高效且鲁棒的3D场景表示对自动驾驶、机器人等领域至关重要。尽管可见光图像提供丰富内容，但热成像或深度等模态可补充环境信息。近期如3D高斯溅射等新视图合成方法开始融合多模态数据以提升性能，但多模态融合常导致速度下降并引入新挑战。为此，本研究致力于基于热红外单一模态，尽可能减少对可见光的依赖。提出热成像转深度高斯溅射（TDg）方法，仅使用热图像与深度估计构建辐射场。在RGBT-Scenes和ThermalMix数据集上，TDg普遍优于基线MSMG，平均渲染质量指标如学习感知图像块相似度（LPIPS）、结构相似性指数（SSIM）、峰值信噪比（PSNR）分别提升1.12%、0.034%、0.01%。同时训练时间显著缩短12分47秒（55%改进）。该方法成功实现了热辐射场建模，适用于监控、搜救及工业设备温度检测等需识别热源的应用。
原文摘要 · Abstract (English)
Efficient and robust 3D scene representation is crucial in autonomous driving, robotics, and related fields. While RGB images provide valuable content for 3D reconstruction, other modalities like thermal or depth can enable additional information on the environment. Lately, novel view synthesis methods like 3D Gaussian Splatting have started using multiple modalities to further boost their performance. But fusing or combining multimodal data can make the process slower and can bring in additional challenges. Therefore, our project aims to use single modality based on thermal infrared domain, by removing the reliance on visible light as much as possible. This single modality can be expected to be faster as it does not rely on multimodal data. We propose a method, Thermal-to-Depth Gaussian Splatting (TDg), that uses only thermal images and depth estimation in its architecture to derive the radiance fields. Our TDg method outperforms the MSMG (Multiple Single-Modal Gaussians) baseline in most cases on our test datasets, RGBT-Scenes and ThermalMix. On average, the rendering quality metrics such as learned perceptual image patch similarity (LPIPS), structural similarity index measure (SSIM), and peak signal-to-noise ratio (PSNR) of TDg are 1.12%, 0.034%, and 0.01% better than the baseline MSMG values. It also reduces the training time significantly, by 12 mins 47 secs (55% improvement). Overall, our method is successful in deriving these thermal radiance fields, which can ultimately have several applications, such as identifying heat sources critical in surveillance, search or rescue operations, and industrial inspections where temperature is widely used to monitor machines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。