arXiv:2606.23147cs.RO2026-06被引 1

用现成视觉语言动作模型实现智能辅助,让人类少动手少费神。

Assistron: Bayesian Shared Autonomy with Off-the-shelf Vision-Language-Action Models

论文配图:Assistron: Bayesian Shared Autonomy with Off-the-shelf Vision-Language-Action Models
图 1 · 摘自论文原文
  • 基于语言指令驱动视觉语言动作模型执行大范围动作,降低用户负担。
  • 在易出错的接触交互中主动请求人类干预,并通过流匹配调整动作。
  • 无需微调模型,避免行为偏差,适合日常任务的通用辅助场景。

我们提出Assistron,一种共享自主模型,利用视觉语言动作(VLA)模型协助用户完成日常活动。方法基于两大原则:(1) 通过VLA驱动的自主性执行宏观动作,最大限度减少人类认知与体力消耗;(2) 在关键失败点优先邀请人类介入。在用户语音指令驱动下,Assistron使用VLA自主执行宏观伸展轨迹,节省用户精力。在高接触性交互中,由于VLA易失效,Assistron采用阶段感知的交互检测机制,主动请求用户干预,并通过流匹配引导调整VLA动作生成。关键在于,该方法无需对VLA进行微调,保护其广泛的行为先验,防止灾难性遗忘,确保模型不会退化为单一专能。我们在涵盖多样化日常操作技能的多任务场景恢复基准上验证了该方法。实验结果表明,Assistron相比纯自主基线显著提升任务成功率,同时相较传统遥操作大幅降低人类认知与体力负荷,提供了一种可扩展、流畅且无负担的辅助操作范式。代码已开源:https://github.com/mousecpn/Assistron.git。

原文摘要 · Abstract (English)

We propose Assistron, a shared autonomy model that leverages Vision-Language-Action (VLA) models to assist the user in daily activities. Our approach is grounded in two core principles: (1)~minimizing human cognitive and physical effort by leveraging VLA-driven autonomy for macro-movements, and (2)~prioritizing human intervention specifically at critical failure points. Driven by the user's verbal language commands, Assistron utilizes the VLA to autonomously execute macro-reaching trajectories, saving users' effort. In contact-rich interactions where VLAs tend to fail, Assistron employs a phase-aware interaction detection mechanism and solicits the user to intervene, in turn adjusting the VLA's action generation via flow matching guidance. Critically, our formulation eliminates the need for VLA fine-tuning, protecting its broad behavioral priors from catastrophic forgetting and ensuring the model does not become a narrow specialist. We validate our approach on a comprehensive multi-task scene recovery benchmark encompassing diverse daily manipulation skills. Empirical results demonstrate that Assistron significantly improves task success rates over pure autonomous baselines while significantly reducing human cognitive and physical workload compared to traditional teleoperation, offering a scalable, smooth, and effortless paradigm for assistive manipulation. The code is available in https://github.com/mousecpn/Assistron.git.

共享自主视觉语言动作人机协作

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。