开源工业场景多模态动作数据集,支持工人行为分析与安全监控。
OpenMarcie: Dataset for Multimodal Action Recognition in Industrial Environments
- 构建工业环境下的多模态数据集,融合可穿戴设备与摄像头数据。
- 包含37小时多视角数据,覆盖36名参与者在两类任务中的真实操作。
- 适用于动作识别、跨模态对齐等研究,适合智能制造与人机交互领域。
智能工厂利用先进技术优化生产流程并提升效率。通过识别工人活动,可精准量化绩效指标,全面提升效率并保障工人安全。OpenMarcie是目前规模最大的面向制造环境的人类动作监测多模态数据集,包含可穿戴设备和周边摄像头采集的数据。数据基于两个实验设置,共36名参与者:第一项为12人进行半真实自行车拆装任务,无固定规程,促进发散性问题解决;第二项为25名志愿者(24份有效数据)完成3D打印机装配任务,依据制造商说明书获取流程知识,并包含序列化协作装配环节,体现真实制造动态。数据集涵盖超过37小时的自我中心与外部视角、多模态、多位置数据,包含8种数据类型及200多个独立信息通道。该数据集已在三项人类活动识别任务上进行基准测试:动作分类、开放词汇描述生成与跨模态对齐。
原文摘要 · Abstract (English)
Smart factories use advanced technologies to optimize production and increase efficiency. To this end, the recognition of worker activity allows for accurate quantification of performance metrics, improving efficiency holistically while contributing to worker safety. OpenMarcie is, to the best of our knowledge, the biggest multimodal dataset designed for human action monitoring in manufacturing environments. It includes data from wearables sensing modalities and cameras distributed in the surroundings. The dataset is structured around two experimental settings, involving a total of 36 participants. In the first setting, twelve participants perform a bicycle assembly and disassembly task under semi-realistic conditions without a fixed protocol, promoting divergent and goal-oriented problem-solving. The second experiment involves twenty-five volunteers (24 valid data) engaged in a 3D printer assembly task, with the 3D printer manufacturer's instructions provided to guide the volunteers in acquiring procedural knowledge. This setting also includes sequential collaborative assembly, where participants assess and correct each other's progress, reflecting real-world manufacturing dynamics. OpenMarcie includes over 37 hours of egocentric and exocentric, multimodal, and multipositional data, featuring eight distinct data types and more than 200 independent information channels. The dataset is benchmarked across three human activity recognition tasks: activity classification, open vocabulary captioning, and cross-modal alignment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。