arXiv:2505.13309cs.CV2025-05

首个面向水下机器人的事件相机光流数据集,支持从仿真到现实的直接迁移。

eStonefish-Scenes: A Sim-to-Real Validated and Robot-Centric Event-based Optical Flow Dataset for Underwater Vehicles

  • 用石鱼模拟器生成带珊瑚礁和会动鱼群的水下场景,构建合成事件流数据集。
  • 仅用合成数据训练的模型在真实水下视频上达到0.79像素误差,无需微调。
  • 提供完整处理工具链,适合水下机器人与事件相机研究者使用。

事件相机(EBCs)有望革新水下机器人技术,但缺乏标注的水下事件数据集严重制约了视觉里程计和避障等任务的发展。真实世界事件光流数据集稀缺、采集成本高且多样性不足,现有基准均未覆盖水下应用。为此,我们提出eStonefish-Scenes,一个基于Stonefish模拟器生成的合成事件光流数据集,并公开可定制的水下环境生成流程,包含逼真的珊瑚礁与具备反应式导航行为的生物仿生鱼群。同时,我们开发eWiz库,涵盖事件数据加载、增强、可视化、编码、训练工具、损失函数及评估指标。为验证仿真到现实的迁移能力,我们在室内测试池中搭载DAVIS346混合事件-帧相机的BlueROV2采集真实数据,通过单应性配准获取真值光流,利用蒙特卡洛扰动估计像素级不确定性,并将其纳入评估指标,实现可靠性感知的性能评估。基于ConvGRU的光流网络仅在合成数据上训练,未进行微调即在真实序列上取得0.79像素的不确定性加权平均端点误差,证明该合成数据集能有效支撑水下事件光流的仿真到现实迁移,大幅降低真实数据采集成本。

原文摘要 · Abstract (English)

Event-based cameras (EBCs) are poised to transform underwater robotics, yet the absence of labelled event-based datasets for underwater environments severely limits progress in tasks such as visual odometry and obstacle avoidance. Real-world event-based optical flow datasets are scarce, resource-intensive to collect, and lack diversity, while no prior benchmarks target underwater applications. To bridge this gap, we introduce eStonefish-Scenes, a synthetic event-based optical flow dataset generated using the Stonefish simulator, together with an open data generation pipeline for creating customizable underwater environments featuring realistic coral reefs and biologically inspired schools of fish with reactive navigation behaviours. We also present eWiz, a comprehensive library for event-based data processing, encompassing data loading, augmentation, visualization, encoding, training utilities, loss functions, and evaluation metrics. To validate sim-to-real transferability, we collected real-world data using a DAVIS346 hybrid event-and-frame camera mounted on a BlueROV2 in an indoor testing pool. Ground-truth optical flow was derived via homography-based frame-to-poster registration, and per-pixel uncertainty was estimated through Monte Carlo perturbation of keypoint correspondences. This uncertainty was incorporated into the evaluation metrics, enabling reliability-aware performance assessment. A ConvGRU-based optical flow network, trained exclusively on synthetic eStonefish-Scenes data, was evaluated on the real-world sequences without fine-tuning, achieving an uncertainty-weighted average endpoint error of 0.79 pixels. These results demonstrate that the proposed synthetic dataset effectively supports sim-to-real transfer for underwater event-based optical flow estimation, substantially reducing the need for costly real-world data collection.

事件相机水下机器人光流估计仿真迁移

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。