事件相机联合估计运动与外观，提升精度且更快。
Unsupervised Joint Learning of Optical Flow and Intensity with Event Cameras
- 用单网络同时学习光流和图像强度，利用事件数据内在关联性
- 光流误差降20%，平均角度误差降25%，高动态范围下表现优异
- 无需监督，推理速度更快，适合实时应用
事件相机依赖运动来获取场景外观信息，因此外观与运动天然耦合：要么两者均存在并被记录，要么均未被捕获。以往工作将这两者恢复视为独立任务，违背了事件相机的本质特性，并忽略了它们之间的内在联系。本文提出一种无监督学习框架，通过单一网络联合估计光流（运动）和图像强度（外观）。基于数据生成模型，我们新推导出以光流和图像强度为变量的事件型光度误差，并将其与对比度最大化框架结合，构建全面损失函数，为光流和强度估计提供有效约束。大量实验表明，本方法在光流估计上相比无监督方法降低20%的端点误差（EPE）和25%的平均角度误差（AE），同时在高动态范围场景中实现具有竞争力的强度重建效果。此外，该方法推理时间短于所有其他光流方法及多数图像重建方法，而后者仅输出单一量。项目页面：https://github.com/tub-rip/E2FAI
原文摘要 · Abstract (English)
Event cameras rely on motion to obtain information about scene appearance. This means that appearance and motion are inherently linked: either both are present and recorded in the event data, or neither is captured. Previous works treat the recovery of these two visual quantities as separate tasks, which does not fit with the above-mentioned nature of event cameras and overlooks the inherent relations between them. We propose an unsupervised learning framework that jointly estimates optical flow (motion) and image intensity (appearance) using a single network. From the data generation model, we newly derive the event-based photometric error as a function of optical flow and image intensity. This error is further combined with the contrast maximization framework to form a comprehensive loss function that provides proper constraints for both flow and intensity estimation. Exhaustive experiments show our method's state-of-the-art performance: in optical flow estimation, it reduces EPE by 20% and AE by 25% compared to unsupervised approaches, while delivering competitive intensity estimation results, particularly in high dynamic range scenarios. Our method also achieves shorter inference time than all other optical flow methods and many of the image reconstruction methods, while they output only one quantity. Project page: https://github.com/tub-rip/E2FAI
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。