保护预训练视觉特征，让机器人模型更通用。
Enhancing Generalization in Vision-Language-Action Models by Preserving Pretrained Representations
- 冻结视觉编码器保留预训练特征，仅训练任务适配部分。
- 用字符序列表示连续动作，对齐预训练数据分布。
- 结合机器人示范与视觉语言数据，提升空间推理能力。
从视觉语言模型（VLM）微调而来的视觉-语言-动作（VLA）模型有望利用丰富的预训练表征，构建适用于多样任务和环境的通用机器人。然而，在机器人数据上直接微调常破坏这些表征，限制泛化能力。本文提出一种框架，更好地保留预训练特征的同时适应机器人操作。方法包含三个组件：(i) 双编码器设计，一个冻结的视觉编码器保留预训练特征，另一个可训练以适应任务；(ii) 基于字符串的动作标记器，将连续动作转换为字符序列，与模型预训练领域对齐；(iii) 共训练策略，融合机器人示范与视觉语言数据集，强调空间推理与物体可操作性。在仿真和真实机器人上的评估表明，该方法提升了对视觉扰动的鲁棒性、对新指令和新环境的泛化能力，以及整体任务成功率，优于基线方法。
原文摘要 · Abstract (English)
Vision-language-action (VLA) models finetuned from vision-language models (VLMs) hold the promise of leveraging rich pretrained representations to build generalist robots across diverse tasks and environments. However, direct fine-tuning on robot data often disrupts these representations and limits generalization. We present a framework that better preserves pretrained features while adapting them for robot manipulation. Our approach introduces three components: (i) a dual-encoder design with one frozen vision encoder to retain pretrained features and another trainable for task adaptation, (ii) a string-based action tokenizer that casts continuous actions into character sequences aligned with the model's pretraining domain, and (iii) a co-training strategy that combines robot demonstrations with vision-language datasets emphasizing spatial reasoning and affordances. Evaluations in simulation and on real robots show that our method improves robustness to visual perturbations, generalization to novel instructions and environments, and overall task success compared to baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。