用头戴式事件相机实现动态环境下的人体姿态估计
Event-based Egocentric Human Pose Estimation in Dynamic Environment
- 先估头部姿态，再以之为条件生成身体姿态
- 引入运动分割模块，有效去除干扰动态物体
- 在合成数据集上四项指标优于基线方法
使用前视头戴式相机进行人体姿态估计对体育动作分析、虚拟现实/增强现实以及可穿戴设备人工智能应用至关重要。然而，现有许多方法依赖于RGB相机，在低光环境或运动模糊情况下表现不佳。事件相机具有应对这些挑战的潜力。本文首次提出使用头戴式事件相机进行人体姿态估计的新任务，并提出D-EventEgo框架。该方法首先估计头部姿态，再以头部姿态作为条件生成身体姿态。由于动态物体与背景事件混合会降低头部姿态估计精度，因此引入运动分割模块以去除动态物体并提取背景信息。在基于EgoBody合成的事件数据集上的大量实验表明，该方法在动态环境下四项评估指标优于基线方法。
原文摘要 · Abstract (English)
Estimating human pose using a front-facing egocentric camera is essential for applications such as sports motion analysis, VR/AR, and AI for wearable devices. However, many existing methods rely on RGB cameras and do not account for low-light environments or motion blur. Event-based cameras have the potential to address these challenges. In this work, we introduce a novel task of human pose estimation using a front-facing event-based camera mounted on the head and propose D-EventEgo, the first framework for this task. The proposed method first estimates the head poses, and then these are used as conditions to generate body poses. However, when estimating head poses, the presence of dynamic objects mixed with background events may reduce head pose estimation accuracy. Therefore, we introduce the Motion Segmentation Module to remove dynamic objects and extract background information. Extensive experiments on our synthetic event-based dataset derived from EgoBody, demonstrate that our approach outperforms our baseline in four out of five evaluation metrics in dynamic environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。