arXiv:2608.16264cs.RO2026-08

让激光雷达生成彩色视频,实现全天候无摄像头感知

Cyclops: LiDAR as a Camera That Dreams in Color

论文配图:Cyclops: LiDAR as a Camera That Dreams in Color
图 1 · 摘自论文原文
  • 用预训练模型将稀疏激光数据转为稠密几何特征
  • 通过微分方程快速迁移至彩色分布,生成稳定视频帧
  • 适合夜间或强光环境下依赖激光雷达的机器人视觉

传统机器人感知依赖相机,但在低光照或高动态范围环境中性能下降。激光雷达虽不受光照影响,但数据稀疏且单通道,难以直接用于预训练的RGB视觉模型。本文提出Cyclops框架,将稀疏非重复扫描激光雷达(NRS-LiDAR)强度数据转换为RGB视频,实现全天候无相机感知。首先通过冻结的预训练稠密化模块将稀疏强度投影转化为稠密表征,作为几何先验;再利用隐空间桥接匹配(LBM)与学习到的速度场,在少数常微分方程(ODE)积分步内将稠密强度潜在表示迁移到目标RGB分布。为减少帧间闪烁,引入前一帧上下文并通过时间注意力层,并将速度场建模为由可微终端奖励优化的策略,通过沿ODE轨迹反向传播增强最终输出保真度。大量实验表明,合成的RGB视频(包括近黑暗条件下的)使标准RGB感知模型在语义分割、车道检测和点云着色任务中显著优于激光雷达基线和传统相机,适用于多种光照场景。

原文摘要 · Abstract (English)

Conventionally, robotic perception relies heavily on cameras due to the rich semantic texture they provide. However, their performance degrades significantly in low-light or high-dynamic-range environments. Conversely, while Light Detection and Ranging (LiDAR) captures illumination-invariant geometric and intensity properties, the resulting data are typically single-channel and sparse, creating a significant modality gap when applying vision models pre-trained on RGB datasets. In this paper, we propose Cyclops, a framework that translates sparse Non-Repetitive Scanning LiDAR (NRS-LiDAR) intensity into RGB video, enabling camera-free inference for all-day perception tasks. Our approach first converts sparse LiDAR intensity projections into dense representations via a frozen pre-trained densification module, serving as a geometrically rich source condition. The dense intensity latent is then transported toward the target RGB distribution through Latent Bridge Matching (LBM) with a learned velocity field in a few ODE integration steps. To mitigate inter-frame flickering, we inject prior-frame context via temporal attention layers and further formulate the velocity field as a policy optimized by a differentiable terminal reward that encourages terminal fidelity through backpropagation along the ODE trajectory. Extensive experiments demonstrate that the synthesized RGB, including those generated under near-dark conditions, enable standard RGB-based perception models to substantially outperform both LiDAR baselines and conventional cameras on semantic segmentation, lane detection, and point cloud colorization across diverse lighting conditions.

激光雷达图像生成机器人感知多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。