通过3D手部姿态识别用户,实现无感身份认证。
Person Identification from Egocentric Human-Object Interactions using 3D Hand Pose
- 基于3D手姿的多阶段识别框架,先识物再识交互最后识人。
- 在双手操作数据集上达到97.52%的平均F1分数。
- 模型轻量(<4MB)、推理快(0.1秒),适合实时设备端使用。
人-物交互识别(HOIR)与用户身份识别对增强现实(AR)个性化辅助技术至关重要。该研究提出I2S(Interact2Sign)多阶段框架,通过拍摄者视角视频中的3D手部姿态分析,实现无侵入式用户识别。I2S利用手工提取的3D手姿特征,依次完成物体类别识别、人-物交互识别与用户身份识别。针对3D手姿设计了空间、频率、运动学、朝向等语义化特征,并引入新描述子——跨手空间包络(IHSE)。大量消融实验确定最优特征组合,在基于ARCTIC与H2O数据集构建的双手操作数据集上,用户识别平均F1得分达97.52%。I2S在保持轻量化(<4MB)和0.1秒推理时间的同时,实现业界领先性能,适用于高安全性的实时AR身份认证系统。
原文摘要 · Abstract (English)
Human-Object Interaction Recognition (HOIR) and user identification play a crucial role in advancing augmented reality (AR)-based personalized assistive technologies. These systems are increasingly being deployed in high-stakes, human-centric environments such as aircraft cockpits, aerospace maintenance, and surgical procedures. This research introduces I2S (Interact2Sign), a multi stage framework designed for unobtrusive user identification through human object interaction recognition, leveraging 3D hand pose analysis in egocentric videos. I2S utilizes handcrafted features extracted from 3D hand poses and per forms sequential feature augmentation: first identifying the object class, followed by HOI recognition, and ultimately, user identification. A comprehensive feature extraction and description process was carried out for 3D hand poses, organizing the extracted features into semantically meaningful categories: Spatial, Frequency, Kinematic, Orientation, and a novel descriptor introduced in this work, the Inter-Hand Spatial Envelope (IHSE). Extensive ablation studies were conducted to determine the most effective combination of features. The optimal configuration achieved an impressive average F1-score of 97.52% for user identification, evaluated on a bimanual object manipulation dataset derived from the ARCTIC and H2O datasets. I2S demonstrates state-of-the-art performance while maintaining a lightweight model size of under 4 MB and a fast inference time of 0.1 seconds. These characteristics make the proposed framework highly suitable for real-time, on-device authentication in security-critical, AR-based systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。