构建首个鼠类时空实例分割数据集,融合事件与帧图像提升运动追踪性能。
MouseSIS: A Frames-and-Events Dataset for Space-Time Instance Segmentation of Mice
- 提出时空实例分割新任务,结合事件流与对齐帧进行目标分割
- 包含7只自由活动小鼠的像素级掩码标注,支持复杂运动场景分析
- 验证事件数据可显著提升追踪精度,尤其在快速运动下表现更优
得益于大规模标注数据集,视频中物体的追踪与分割近年来取得显著进展。然而,在图像质量下降或物体快速移动时,现有算法仍表现不佳。事件相机具有高时间分辨率和高动态范围,能有效应对这些挑战。但缺乏用于基于学习的掩码级追踪算法的事件数据标注。为此,我们提出:(i)一项新任务——空间-时间实例分割,类似视频实例分割,目标是在传感器输入全程(此处为连续事件流及可选对齐帧)内分割出每个实例;(ii)�,一个用于该任务的数据集,包含对齐的灰度帧和事件数据。数据集涵盖最多七只自由移动并相互交互的小鼠,提供像素级实例分割掩码标注。我们还提供了两种基准方法,结果表明结合事件数据能持续提升追踪性能,尤其在与传统摄像头联合使用时效果更佳。结果凸显了事件辅助追踪在困难场景下的潜力。我们希望该数据集推动事件基视频实例分割领域发展,助力鲁棒追踪算法在挑战性条件下的研发。
原文摘要 · Abstract (English)
Enabled by large annotated datasets, tracking and segmentation of objects in videos has made remarkable progress in recent years. Despite these advancements, algorithms still struggle under degraded conditions and during fast movements. Event cameras are novel sensors with high temporal resolution and high dynamic range that offer promising advantages to address these challenges. However, annotated data for developing learning-based mask-level tracking algorithms with events is not available. To this end, we introduce: ($i$) a new task termed \emph{space-time instance segmentation}, similar to video instance segmentation, whose goal is to segment instances throughout the entire duration of the sensor input (here, the input are quasi-continuous events and optionally aligned frames); and ($ii$) \emph{\dname}, a dataset for the new task, containing aligned grayscale frames and events. It includes annotated ground-truth labels (pixel-level instance segmentation masks) of a group of up to seven freely moving and interacting mice. We also provide two reference methods, which show that leveraging event data can consistently improve tracking performance, especially when used in combination with conventional cameras. The results highlight the potential of event-aided tracking in difficult scenarios. We hope our dataset opens the field of event-based video instance segmentation and enables the development of robust tracking algorithms for challenging conditions.\url{https://github.com/tub-rip/MouseSIS}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。