arXiv:2510.05213cs.ROcs.AI2025-10被引 3

用专家库动态选视觉模型,让机器人学得更快更准

VER: Vision Expert Transformer for Robot Learning via Foundation Distillation and Dynamic Routing

  • 预训练多个视觉模型,构建专家库,仅微调少量路由网络
  • 在17个任务上表现领先,减少无关区域的干扰
  • 适合需要快速适配新任务的机器人系统

预训练视觉基础模型(VFMs)通过丰富的视觉表征推动机器人学习,但单一模型通常只在特定领域表现优异,限制了跨任务通用性。将多个VFMs融合为统一策略表示可缓解此问题,但常导致任务特定特征选择僵化,且需昂贵的全量重训练来融入机器人领域知识。我们提出VER:一种面向机器人学习的视觉专家变换器。预训练阶段,VER将多个VFMs提炼为视觉专家库;微调时仅更新轻量级路由网络(参数少于0.4%),动态从预训练专家库中选择任务相关专家。我们进一步引入逐块专家路由与课程式Top-K退火机制,提升动态选择的灵活性与精度。此外,VER支持参数高效微调,实现专家的可扩展利用与自适应领域知识融合。在17个多样化机器人任务和多个策略头下,VER达到当前最优性能。我们发现,VER能有效降低无关区域(如背景)的大范数异常值,聚焦于任务关键区域。可视化与代码见https://yixiaowang7.github.io/ver_page/。

原文摘要 · Abstract (English)

Pretrained vision foundation models (VFMs) advance robotic learning via rich visual representations, yet individual VFMs typically excel only in specific domains, limiting generality across tasks. Distilling multiple VFMs into a unified representation for policy can mitigate this limitation but often yields inflexible task-specific feature selection and requires costly full re-training to incorporate robot-domain knowledge. We propose VER, a Vision Expert transformer for Robot learning. During pretraining, VER distills multiple VFMs into a vision expert library. It then fine-tunes only a lightweight routing network (fewer than 0.4% of parameters) to dynamically select task-relevant experts from the pretrained library for downstream robot tasks. We further introduce Patchwise Expert Routing with Curriculum Top-K Annealing to improve both flexibility and precision of dynamic expert selection. Moreover, VER supports parameter-efficient finetuning for scalable expert utilization and adaptive robot-domain knowledge integration. Across 17 diverse robotic tasks and multiple policy heads, VER achieves state-of-the-art performance. We find that VER reduces large-norm outliers in task-irrelevant regions (e.g., background) and concentrates on task-critical regions. Visualizations and codes can be found in https://yixiaowang7.github.io/ver_page/.

机器人学习视觉专家动态路由参数高效

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。