用热成像与可见光融合，实时清除复杂烟雾并重建3D场景。
SmokeSeer: 3D Gaussian Splatting for Smoke Removal and Scene Reconstruction
- 融合热成像与可见光，利用热辐射穿透烟雾的特性
- 在合成数据和新实测数据集上实现高密度动态烟雾清除
- 适合火灾救援、工业监测等需穿烟视觉的场景
真实场景中的烟雾会严重降低图像质量并影响能见度。现有图像恢复方法或依赖易产生幻觉的数据驱动先验，或仅适用于静态低密度烟雾。我们提出SmokeSeer，一种从多视角视频序列中同时进行3D场景重建与烟雾去除的方法。该方法利用热成像与可见光图像，借助热成像在烟雾中散射更少的特性实现穿透。基于3D Gaussian splatting框架，融合双模态信息，将场景分解为烟雾与非烟雾成分。与以往工作不同，SmokeSeer可处理多种烟雾密度，并适应随时间变化的烟雾。我们在合成数据及一个新的包含RGB与热成像的真实世界烟雾数据集上验证了该方法的有效性。项目提供开源代码与数据集。
原文摘要 · Abstract (English)
Smoke in real-world scenes can severely degrade image quality and hamper visibility. Recent image restoration methods either rely on data-driven priors that are susceptible to hallucinations, or are limited to static low-density smoke. We introduce SmokeSeer, a method for simultaneous 3D scene reconstruction and smoke removal from multi-view video sequences. Our method uses thermal and RGB images, leveraging the reduced scattering in thermal images to see through smoke. We build upon 3D Gaussian splatting to fuse information from the two image modalities, and decompose the scene into smoke and non-smoke components. Unlike prior work, SmokeSeer handles a broad range of smoke densities and adapts to temporally varying smoke. We validate our method on synthetic data and a new real-world smoke dataset with RGB and thermal images. We provide an open-source implementation and data on the project website.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。