提出轻量图序列编码器，实现实时操作动作识别并有效处理长时序数据。
Real-Time Manipulation Action Recognition with a Factorized Graph Sequence Encoder
- 采用分解式编码结构，提升时间维度上的可扩展性。
- 在Bimacs和CoAx数据集上F1-macro分别提升14.3%和5.6%。
- 适合需要实时、长时序动作识别的机器人协作场景。
实时识别人类操作动作对人机安全交互至关重要。现有方法虽能实时运行，但在长时序操作下难以有效适应。为此，本文基于通用场景图表示，提出一种分解式图序列编码器，兼具实时性与时间可扩展性。引入手部池化操作，更聚焦地提取图级嵌入。模型在KIT Bimanual Action (Bimacs) 和 Collaborative Action (CoAx) 数据集上，分别实现F1-macro 14.3% 和 5.6% 的提升，优于当前最优实时方法。通过消融实验验证设计有效性，并对比了同架构的RGB模型，证明其在以物体为中心的操作任务中更具优势。
原文摘要 · Abstract (English)
Recognition of human manipulation actions in real-time is essential for safe and effective human-robot interaction and collaboration. The challenge lies in developing a model that is both lightweight enough for real-time execution and capable of generalization. While some existing methods in the literature can run in real-time, they struggle with temporal scalability, i.e., they fail to adapt to long-duration manipulations effectively. To address this, leveraging the generalizable scene graph representations, we propose a new Factorized Graph Sequence Encoder network that not only runs in real-time but also scales effectively in the temporal dimension, thanks to its factorized encoder architecture. Additionally, we introduce Hand Pooling operation, a simple pooling operation for more focused extraction of the graph-level embeddings. Our model outperforms the previous state-of-the-art real-time approach, achieving a 14.3\% and 5.6\% improvement in F1-macro score on the KIT Bimanual Action (Bimacs) Dataset and Collaborative Action (CoAx) Dataset, respectively. Moreover, we conduct an extensive ablation study to validate our network design choices. Finally, we compare our model with its architecturally similar RGB-based model on the Bimacs dataset and show the limitations of this model in contrast to ours on such an object-centric manipulation dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。