arXiv:2409.13792cs.LGcs.AI2024-09中稿 · Wiley Advanced Rob…被引 3

让软夹爪持续学习多模态数据,边学边用不丢旧知识。

Continual Learning for Multimodal Data Fusion of a Soft Gripper

  • 融合触觉与视觉数据,通过原型存储实现增量学习。
  • 在自定义数据集上达到92.3%准确率,优于基线模型。
  • 适合机器人感知、少标注场景的在线学习应用。

持续学习(CL)指算法能持续从环境中增量获取新知识并保留旧知识的能力。单一模态训练的模型在面对新模态时表现下降。传统方法通过拼接特征融合,但需重新训练。本文提出一种新型持续学习算法,结合类别增量与领域增量学习,在标签稀缺但非独立同分布的无标签数据丰富的仿真环境中,仅需存储每类原型即可高效学习。在包含软气动夹爪触觉数据和视频序列中提取的非静态物体视觉数据的定制多模态数据集上验证有效性,并在自定义数据集与Core50数据集上进行消融实验,验证各模块贡献。进一步通过同步于ROS框架的真实机器人系统,实现夹爪与外置摄像头联合实时物体分类,验证算法鲁棒性。

原文摘要 · Abstract (English)

Continual learning (CL) refers to the ability of an algorithm to continuously and incrementally acquire new knowledge from its environment while retaining previously learned information. A model trained on one data modality often fails when tested with a different modality. A straightforward approach might be to fuse the two modalities by concatenating their features and training the model on the fused data. However, this requires retraining the model from scratch each time it encounters a new domain. In this paper, we introduce a continual learning algorithm capable of incrementally learning different data modalities by leveraging both class-incremental and domain-incremental learning scenarios in an artificial environment where labeled data is scarce, yet non-iid (independent and identical distribution) unlabeled data from the environment is plentiful. The proposed algorithm is efficient and only requires storing prototypes for each class. We evaluate the algorithm's effectiveness on a challenging custom multimodal dataset comprising of tactile data from a soft pneumatic gripper, and visual data from non-stationary images of objects extracted from video sequences. Additionally, we conduct an ablation study on the custom dataset and the Core50 dataset to highlight the contributions of different components of the algorithm. To further demonstrate the robustness of the algorithm, we perform a real-time experiment for object classification using the soft gripper and an external independent camera setup, all synchronized with the Robot Operating System (ROS) framework.

持续学习多模态融合机器人感知原型学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。