arXiv:2603.06084cs.RO2026-03被引 1

用小模型生成机器人任务行为树,效果接近大厂闭源方案。

Multimodal Behavior Tree Generation: A Small Vision-Language Model for Robot Task Planning

  • 用小规模视觉语言模型结合行为树框架,实现从视觉指令到可执行任务的自动规划。
  • 在仿真环境中完成家务任务,40亿参数模型成功率87%,仅需少量计算资源。
  • 自建数据集+高效微调,适合研究机器人自主决策与轻量化部署的团队使用。

大型与小型语言模型已被广泛用于机器人任务规划。与此同时,视觉语言模型(VLMs)在图像描述、场景理解及视觉问答等任务中表现优异。本文将二者结合,采用一个紧凑且开源的多模态模型,生成适用于机器人任务规划的行为树。主要挑战在于缺乏将视觉观测与指令映射为可执行行为树的数据集。为此,我们基于现有机器人任务数据集(Open X-Embodiment),通过大型模型作为教师,在多阶段生成流程中构建该数据集。利用此数据集,我们对500M至40亿参数的VLMs进行参数高效微调(PEFT)。生成的行为树兼容BehaviorTree.CPP库,通过结构与词汇指标进行离线评估,并在先进的具身模拟器中执行家庭任务进行在线验证。结果表明,经过微调的40亿参数VLM达到87%的成功率,性能接近当前最优闭源模型,同时所需计算资源仅为极小部分。

原文摘要 · Abstract (English)

Large and small language models have been widely used for robotic task planning. At the same time, vision-language models (VLMs) have successfully tackled problems such as image captioning, scene understanding, and visual question answering. In this work, we combine these two approaches by deploying a compact, open-source multimodal model to generate behavior trees for robotic task planning. The main obstacle to achieving this goal is the lack of an existing dataset that links visual observations and instructions to executable behavior trees. We propose a method to construct such a dataset starting from existing robotic episodes (i.e., Open X-Embodiment), in which a large model serves as a teacher in a multi-stage generation pipeline. We use this dataset to fine-tune VLMs ranging from 500M to 4B parameters via parameter-efficient fine-tuning (PEFT). The generated behavior trees, compatible with the BehaviorTree.CPP library, are evaluated both offline, using structural and lexical metrics, and online through the execution of household tasks in a state-of-the-art embodied simulator. Our results demonstrate that our fine-tuned 4B-parameter VLM approaches the performance of state-of-the-art closed-source models, achieving an 87\% success rate while requiring only a fraction of the computational resources.

机器人规划行为树多模态模型轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。