构建首个高保真多模态急救数据集,助力AI辅助急救决策。
EgoEMS: A High-Fidelity Multimodal Egocentric Dataset for Cognitive Assistance in Emergency Medical Services
- 采集62人参与的233个模拟急救场景,含20小时第一视角多模态数据。
- 标注关键步骤、语音转录、动作质量评分及目标框与分割掩码。
- 提供实时多模态关键步骤识别与动作质量评估基准,适合急救AI研究者。
急救服务在紧急情况下对患者生存至关重要,但一线人员常面临高强度认知负荷。人工智能认知助手作为虚拟伙伴,有望通过支持实时数据采集与决策来减轻负担。为此,我们推出EgoEMS,首个端到端、高保真、多模态、多人员数据集,涵盖233个模拟急救场景中62名参与者(含46名急救专业人员)超过20小时的真实急救活动,采用第一视角拍摄。该数据集由急救专家协作开发,符合国家标准,使用开源、低成本、可复现的数据采集系统,并标注了关键步骤、带说话人分离的时序语音转录、动作质量指标,以及带边界框和分割掩码的视觉信息。强调真实感,包含响应者与患者互动的真实应急动态。我们还提出一套用于实时多模态关键步骤识别与动作质量估计的基准,对开发急救领域人工智能支持工具至关重要。期望EgoEMS能激发研究社区推动智能急救系统发展,最终提升患者救治效果。
原文摘要 · Abstract (English)
Emergency Medical Services (EMS) are critical to patient survival in emergencies, but first responders often face intense cognitive demands in high-stakes situations. AI cognitive assistants, acting as virtual partners, have the potential to ease this burden by supporting real-time data collection and decision making. In pursuit of this vision, we introduce EgoEMS, the first end-to-end, high-fidelity, multimodal, multiperson dataset capturing over 20 hours of realistic, procedural EMS activities from an egocentric view in 233 simulated emergency scenarios performed by 62 participants, including 46 EMS professionals. Developed in collaboration with EMS experts and aligned with national standards, EgoEMS is captured using an open-source, low-cost, and replicable data collection system and is annotated with keysteps, timestamped audio transcripts with speaker diarization, action quality metrics, and bounding boxes with segmentation masks. Emphasizing realism, the dataset includes responder-patient interactions reflecting real-world emergency dynamics. We also present a suite of benchmarks for real-time multimodal keystep recognition and action quality estimation, essential for developing AI support tools for EMS. We hope EgoEMS inspires the research community to push the boundaries of intelligent EMS systems and ultimately contribute to improved patient outcomes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。