arXiv:2503.03890cs.ROcs.LG2025-03

用语言增强的稀疏特征蒸馏,实现单视角少样本灵巧抓取。

LensDFF: Language-enhanced Sparse Feature Distillation for Efficient Few-Shot Dexterous Manipulation

  • 通过语言引导融合2D视觉特征到3D点,实现高效特征蒸馏。
  • 在真实与仿真环境中均超越现有方法,抓取成功率更高。
  • 适合做少样本灵巧操作研究或机器人抓取系统开发人员。

从少量示范中学习灵巧操作是先进类人机器人系统的重要挑战。现有密集特征场方法虽能从2D视觉基础模型中蒸馏丰富语义特征至3D空间,但依赖NeRF或Gaussian Splatting等神经渲染模型,计算开销大。以往基于稀疏特征场的方法或因多视角依赖导致效率低,或缺乏足够的抓取灵活性。为此,本文提出语言增强型稀疏蒸馏特征场(LensDFF),通过创新的语言增强特征融合策略,将一致的2D特征高效映射至3D点,实现单视角少样本泛化。基于LensDFF,进一步构建少样本灵巧操作框架,结合抓取基元生成稳定且高灵巧性的抓取动作。同时设计真实到仿真(real2sim)抓取评估流水线,支持高效评估与超参数调优。在基于real2sim流水线的大量仿真实验及真实世界测试中,本方法表现优异,优于当前最优方法。

原文摘要 · Abstract (English)

Learning dexterous manipulation from few-shot demonstrations is a significant yet challenging problem for advanced, human-like robotic systems. Dense distilled feature fields have addressed this challenge by distilling rich semantic features from 2D visual foundation models into the 3D domain. However, their reliance on neural rendering models such as Neural Radiance Fields (NeRF) or Gaussian Splatting results in high computational costs. In contrast, previous approaches based on sparse feature fields either suffer from inefficiencies due to multi-view dependencies and extensive training or lack sufficient grasp dexterity. To overcome these limitations, we propose Language-ENhanced Sparse Distilled Feature Field (LensDFF), which efficiently distills view-consistent 2D features onto 3D points using our novel language-enhanced feature fusion strategy, thereby enabling single-view few-shot generalization. Based on LensDFF, we further introduce a few-shot dexterous manipulation framework that integrates grasp primitives into the demonstrations to generate stable and highly dexterous grasps. Moreover, we present a real2sim grasp evaluation pipeline for efficient grasp assessment and hyperparameter tuning. Through extensive simulation experiments based on the real2sim pipeline and real-world experiments, our approach achieves competitive grasping performance, outperforming state-of-the-art approaches.

灵巧操作少样本学习特征蒸馏语言增强

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。