arXiv:2411.12250cs.CVcs.RO2024-11被引 7

让视频转事件更真实,模拟了像素电路的模拟特性。

ADV2E: Bridging the Gap Between Analogue Circuit and Discrete Frames in the Video-to-Events Simulator

  • 基于事件相机像素电路的模拟特性设计新仿真方法
  • 在高对比度场景下仍能生成可靠事件数据
  • 适合训练深度神经网络的事件数据生成

事件相机的工作原理与传统主动像素传感器(APS)相机截然不同,具有显著优势。近年来的研究开发了将视频帧转换为事件的仿真器,以解决真实事件数据集稀缺的问题。现有仿真器主要关注事件相机的逻辑行为,但极少考虑像素电路的根本模拟特性。模拟电路与离散视频帧之间的差距导致合成事件质量下降,尤其在高对比度场景中更为明显。本文提出一种新方法,通过深入分析事件相机像素电路,将模拟特性融入仿真器设计:(1) 从光强信号到事件的模拟滤波,(2) 截止频率独立于视频帧率。在语义分割和图像重建两个任务上的实验验证了合成事件数据的可靠性,即使在高对比度场景中亦然。结果表明,深度神经网络能从仿真数据有效泛化到真实事件数据,证明本文方法生成的事件数据既真实又适用于有效训练。

原文摘要 · Abstract (English)

Event cameras operate fundamentally differently from traditional Active Pixel Sensor (APS) cameras, offering significant advantages. Recent research has developed simulators to convert video frames into events, addressing the shortage of real event datasets. Current simulators primarily focus on the logical behavior of event cameras. However, the fundamental analogue properties of pixel circuits are seldom considered in simulator design. The gap between analogue pixel circuit and discrete video frames causes the degeneration of synthetic events, particularly in high-contrast scenes. In this paper, we propose a novel method of generating reliable event data based on a detailed analysis of the pixel circuitry in event cameras. We incorporate the analogue properties of event camera pixel circuits into the simulator design: (1) analogue filtering of signals from light intensity to events, and (2) a cutoff frequency that is independent of video frame rate. Experimental results on two relevant tasks, including semantic segmentation and image reconstruction, validate the reliability of simulated event data, even in high-contrast scenes. This demonstrates that deep neural networks exhibit strong generalization from simulated to real event data, confirming that the synthetic events generated by the proposed method are both realistic and well-suited for effective training.

事件相机仿真器像素电路视频转事件

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。