构建首个室内布局动态估计的事件相机数据集,支持高动态场景建模。
Ev-Layout: A Large-scale Event-based Multi-modal Dataset for Indoor Layout Estimation and Tracking
- 融合RGB、事件相机与IMU数据,捕捉移动中室内布局变化
- 包含771.3万张图像与100亿事件点,39,000张带布局标注
- 提出事件时序分布特征模块,提升动态环境下的布局精度
本文提出Ev-Layout,一个面向室内布局估计与追踪的大规模事件相机多模态数据集。该数据集通过头戴式显示设备与虚拟现实界面,集成RGB相机与类生物事件相机,实现运动状态下的室内布局采集。同步记录惯性测量单元(IMUs)与环境光照的时间序列数据,揭示运动速度与光照对布局估计精度的影响。数据集包含2.5千段序列,涵盖超过771.3万张RGB图像与100亿个事件数据点,其中39,000张图像附带室内布局标注,支持事件与视频双模态研究。基于此,我们提出一种事件相机布局估计流程,引入新颖的事件-时序分布特征模块以有效聚合时空信息,并设计可嵌入Transformer的时空特征融合模块。在Ev-Layout上进行基准测试与大量实验,结果表明所提方法显著优于现有事件基方法,提升了动态室内布局估计的准确性。
原文摘要 · Abstract (English)
This paper presents Ev-Layout, a novel large-scale event-based multi-modal dataset designed for indoor layout estimation and tracking. Ev-Layout makes key contributions to the community by: Utilizing a hybrid data collection platform (with a head-mounted display and VR interface) that integrates both RGB and bio-inspired event cameras to capture indoor layouts in motion. Incorporating time-series data from inertial measurement units (IMUs) and ambient lighting conditions recorded during data collection to highlight the potential impact of motion speed and lighting on layout estimation accuracy. The dataset consists of 2.5K sequences, including over 771.3K RGB images and 10 billion event data points. Of these, 39K images are annotated with indoor layouts, enabling research in both event-based and video-based indoor layout estimation. Based on the dataset, we propose an event-based layout estimation pipeline with a novel event-temporal distribution feature module to effectively aggregate the spatio-temporal information from events. Additionally, we introduce a spatio-temporal feature fusion module that can be easily integrated into a transformer module for fusion purposes. Finally, we conduct benchmarking and extensive experiments on the Ev-Layout dataset, demonstrating that our approach significantly improves the accuracy of dynamic indoor layout estimation compared to existing event-based methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。