arXiv:2603.02623cs.ROcs.LG2026-03中稿 · ICRA被引 4

让机器人自动学习新技能,不用人工标注就能适应新任务。

Uni-Skill: Building Self-Evolving Skill Repository for Generalizable Robotic Manipulation

  • 用自动生成的技能库替代固定技能,支持动态扩展。
  • 在仿真和真实场景中均实现领先性能,零样本泛化能力强。
  • 适合需要持续学习新操作的智能机器人系统研究者。

现有以技能为中心的方法依赖预定义技能库,在面对新任务时难以自适应。为此,我们提出Uni-Skill框架,支持技能感知规划与自动技能演化。当现有技能不足时,该框架会主动请求新技能实现,从而实现技能库的自我扩充。为支持规划模块对多样技能的自动实现,我们构建了SkillFolder——一个基于大规模未结构化机器人视频、受VerbNet启发的分层技能仓库。通过自动标注大量演示数据填充该分类体系,实现了从低效人工标注到高效离线结构化检索的范式转变。检索到的示例提供行为模式的语义监督与精细的空间轨迹参考,支持仅需少量样本即可推断新技能,无需部署时额外演示。在仿真与真实场景中的全面实验验证了其优于现有视觉语言模型(VLM)基技能方法的性能,展现出强大的推理能力和广泛的零样本泛化能力。

原文摘要 · Abstract (English)

While skill-centric approaches leverage foundation models to enhance generalization in compositional tasks, they often rely on fixed skill libraries, limiting adaptability to new tasks without manual intervention. To address this, we propose Uni-Skill, a Unified Skill-centric framework that supports skill-aware planning and facilitates automatic skill evolution. Unlike prior methods that restrict planning to predefined skills, Uni-Skill requests for new skill implementations when existing ones are insufficient, ensuring adaptable planning with self-augmented skill library. To support automatic implementation of diverse skills requested by the planning module, we construct SkillFolder, a VerbNet-inspired repository derived from large-scale unstructured robotic videos. SkillFolder introduces a hierarchical skill taxonomy that captures diverse skill descriptions at multiple levels of abstraction. By populating this taxonomy with large-scale, automatically annotated demonstrations, Uni-Skill shifts the paradigm of skill acquisition from inefficient manual annotation to efficient offline structural retrieval. Retrieved examples provide semantic supervision over behavior patterns and fine-grained references for spatial trajectories, enabling few-shot skill inference without deployment-time demonstrations. Comprehensive experiments in both simulation and real-world settings verify the state-of-the-art performance of Uni-Skill over existing VLM-based skill-centric approaches, highlighting its advanced reasoning capabilities and strong zero-shot generalization across a wide range of novel tasks.

机器人操作技能演化自监督学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。