用静态RNN编码器实现97.6%手物交互分类准确率
Beyond Sequences: A Benchmark for Atomic Hand-Object Interaction Using a Static RNN Encoder
- 将视频转为动态特征向量,用单步RNN当静态编码器
- 在27,476个样本上达97.6%准确率,抓取类F1达0.90
- 适合做低层交互识别的轻量级模型研究者参考
可靠预测手物交互中的人类意图是计算机视觉的开放挑战。本研究聚焦基础子问题:细粒度分类原子交互状态——'接近'、'抓取'和'持握'。我们设计结构化数据工程流程,将MANIAC数据集原始视频转换为27,476个统计-运动特征向量,每个向量捕捉短时窗内的关系与动态属性。初始假设认为序列建模至关重要,因此对比了静态分类器(MLP)与时序模型(RNN)。出人意料的是,当双向RNN序列长度设为1(seq_length=1)时,网络功能转变为高容量静态特征编码器,直接带来显著准确率提升,最终达到97.60%。尤其值得注意的是,优化模型成功克服最困难的过渡类别'抓取',实现了0.90的平衡F1分数。这些发现为使用结构化可解释特征与轻量架构的手物交互识别提供了新基准。
原文摘要 · Abstract (English)
Reliably predicting human intent in hand-object interactions is an open challenge for computer vision. Our research concentrates on a fundamental sub-problem: the fine-grained classification of atomic interaction states, namely 'approaching', 'grabbing', and 'holding'. To this end, we introduce a structured data engineering process that converts raw videos from the MANIAC dataset into 27,476 statistical-kinematic feature vectors. Each vector encapsulates relational and dynamic properties from a short temporal window of motion. Our initial hypothesis posited that sequential modeling would be critical, leading us to compare static classifiers (MLPs) against temporal models (RNNs). Counter-intuitively, the key discovery occurred when we set the sequence length of a Bidirectional RNN to one (seq_length=1). This modification converted the network's function, compelling it to act as a high-capacity static feature encoder. This architectural change directly led to a significant accuracy improvement, culminating in a final score of 97.60%. Of particular note, our optimized model successfully overcame the most challenging transitional class, 'grabbing', by achieving a balanced F1-score of 0.90. These findings provide a new benchmark for low-level hand-object interaction recognition using structured, interpretable features and lightweight architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。