实时检测工业场景中第一人称视角的手物交互，兼顾速度与精度。
A Real-Time System for Egocentric Hand-Object Interaction Detection in Industrial Domains
- 分步检测：先识别动作再定位物体，提升效率
- 30帧下手物交互检测达38.52% p-AP，手物检测准确率85.13%
- 适合工业质检、智能助手等需快速响应的场景
手物交互检测在实时应用中仍具挑战，其用户体验依赖于对周围物体交互的快速准确识别。本文提出一种基于流式第一人称视觉的高效手物交互检测方法，采用级联架构，包含动作识别与物体检测双模块。动作识别使用以EfficientNetV2为骨干的Mamba模型，在ENIGMA-51基准上实现30fps下的38.52% p-AP；物体检测采用微调后的YOLOWorld，在手物检测任务中达到85.13% AP。当动作识别预测出接触状态时，触发物体检测模块对相关帧进行推理，以定位并分类活跃物体。
原文摘要 · Abstract (English)
Hand-object interaction detection remains an open challenge in real-time applications, where intuitive user experiences depend on fast and accurate detection of interactions with surrounding objects. We propose an efficient approach for detecting hand-objects interactions from streaming egocentric vision that operates in real time. Our approach consists of an action recognition module and an object detection module for identifying active objects upon confirmed interaction. Our Mamba model with EfficientNetV2 as backbone for action recognition achieves 38.52% p-AP on the ENIGMA-51 benchmark at 30fps, while our fine-tuned YOLOWorld reaches 85.13% AP for hand and object. We implement our models in a cascaded architecture where the action recognition and object detection modules operate sequentially. When the action recognition predicts a contact state, it activates the object detection module, which in turn performs inference on the relevant frame to detect and classify the active object.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。