解决热红外图像新视角合成中的模糊与伪影问题
Veta-GS: View-dependent deformable 3D Gaussian Splatting for thermal infrared Novel-view Synthesis
- 引入视角依赖的形变场，捕捉热辐射细微变化
- 在TI-NSD数据集上优于现有方法，显著减少模糊和伪影
- 适合热红外视觉、自动驾驶等需高鲁棒性的场景
基于热红外（TIR）成像的3D高斯点云渲染（3D-GS）近年来在新视角合成中受到关注，实现实时渲染。然而，热红外图像的新视角合成受限于传输效应、发射率差异及低分辨率，导致渲染图像出现浮点和模糊现象。为此，本文提出Veta-GS，通过视角依赖形变场与热特征提取器（TFE）精准建模微小热变化并增强鲁棒性。具体地，形变场结合相机位置与视角方向，有效捕捉热分布变化；同时引入TFE与单目结构相似性（MonoSSIM）损失，综合考虑外观、边缘与频率特性。在TI-NSD基准测试中，本方法性能优于现有方法。
原文摘要 · Abstract (English)
Recently, 3D Gaussian Splatting (3D-GS) based on Thermal Infrared (TIR) imaging has gained attention in novel-view synthesis, showing real-time rendering. However, novel-view synthesis with thermal infrared images suffers from transmission effects, emissivity, and low resolution, leading to floaters and blur effects in rendered images. To address these problems, we introduce Veta-GS, which leverages a view-dependent deformation field and a Thermal Feature Extractor (TFE) to precisely capture subtle thermal variations and maintain robustness. Specifically, we design view-dependent deformation field that leverages camera position and viewing direction, which capture thermal variations. Furthermore, we introduce the Thermal Feature Extractor (TFE) and MonoSSIM loss, which consider appearance, edge, and frequency to maintain robustness. Extensive experiments on the TI-NSD benchmark show that our method achieves better performance over existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。