arXiv:2502.10720cs.CVcs.GR2025-02被引 1

用白天图像生成逼真夜间图像,提升自动驾驶视觉系统夜间表现

NPSim: Nighttime Photorealistic Simulation From Daytime Images With Monocular Inverse Rendering and Ray Tracing

  • 从单张白天图像重建场景网格,融合语义与几何信息
  • 通过真实夜间光照和材质模拟复杂低光交互效果
  • 适用于多数据集,可支撑算法训练与人类感知评估

语义分割是自动驾驶的关键任务,系统需应对全天候条件,包括夜间。构建准确且多样化的夜间语义分割数据集对提升计算机视觉在低光条件下的性能至关重要。本文提出NPSim方法,利用单目逆渲染与光线追踪技术,从真实白天图像生成逼真夜间图像。NPSim包含两个核心组件:网格重建与重光照。网格重建模块结合输入RGB图像的几何信息与对应语义标签的语义信息,生成精确的场景结构表示。重光照模块整合真实夜间光源与材质特性,模拟低光条件下光与物体表面的复杂交互。本研究主要聚焦网格重建组件的实现与评估。实验表明该组件能生成高质量场景网格,并在多个自动驾驶数据集上具有通用性。此外,本文提出了完整的评估方案,涵盖先进监督与无监督分割模型的定量指标,以及人类感知实验,旨在验证所生成夜间图像的真实感,及其数据集对未来研究的推动价值。

原文摘要 · Abstract (English)

Semantic segmentation is an important task for autonomous driving. A powerful autonomous driving system should be capable of handling images under all conditions, including nighttime. Generating accurate and diverse nighttime semantic segmentation datasets is crucial for enhancing the performance of computer vision algorithms in low-light conditions. In this thesis, we introduce a novel approach named NPSim, which enables the simulation of realistic nighttime images from real daytime counterparts with monocular inverse rendering and ray tracing. NPSim comprises two key components: mesh reconstruction and relighting. The mesh reconstruction component generates an accurate representation of the scene structure by combining geometric information extracted from the input RGB image and semantic information from its corresponding semantic labels. The relighting component integrates real-world nighttime light sources and material characteristics to simulate the complex interplay of light and object surfaces under low-light conditions. The scope of this thesis mainly focuses on the implementation and evaluation of the mesh reconstruction component. Through experiments, we demonstrate the effectiveness of the mesh reconstruction component in producing high-quality scene meshes and their generality across different autonomous driving datasets. We also propose a detailed experiment plan for evaluating the entire pipeline, including both quantitative metrics in training state-of-the-art supervised and unsupervised semantic segmentation approaches and human perceptual studies, aiming to indicate the capability of our approach to generate realistic nighttime images and the value of our dataset in steering future progress in the field.

夜视仿真逆渲染自动驾驶图像生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。