arXiv:2605.08879cs.RO2026-05被引 1

提出新方法防止视觉语言动作模型微调时遗忘预训练能力

Preserving Foundational Capabilities in Flow-Matching VLAs through Conservative SFT

论文配图:Preserving Foundational Capabilities in Flow-Matching VLAs through Conservative SFT
图 1 · 摘自论文原文
  • 根据模型置信度动态调整学习信号,抑制低置信样本的过度更新
  • 在多个基准上比普通微调保留能力高出20%以上,无需额外数据
  • 适合需要稳定迁移能力的机器人任务,尤其看重泛化与鲁棒性

无约束微调流匹配视觉-语言-动作(VLA)模型会导致参数密集重写,损害预训练能力。我们提出保守监督微调(ConSFT),一种自适应目标分布且缓解灾难性遗忘的优化方法,无需先验数据或架构开销。通过基于模型置信度动态缩放学习信号,ConSFT 抑制低置信样本的过大致梯度,从而控制内在参数扰动风险。受强化学习信任域裁剪启发,该方法建立渐进式学习动态,确保目标收敛与先验能力保留,仅需稀疏参数更新,不依赖显式正则化所需的并行参考网络。我们在 LIBERO 与 RoboTwin 基准上评估了 ConSFT 在主流流匹配 VLA($π_0$, $π_{0.5}$, GR00T-N1.6-3B)上的表现。结果表明,该方法在能力保留上比普通 SFT 平均提升超 20%绝对值,媲美依赖大量经验回放的方法,在无先验数据条件下实现同等效果。真实机器人部署验证显示,ConSFT 可避免下游适配中的空间过拟合,保持预训练物理技能的同时习得序列目标任务。

原文摘要 · Abstract (English)

Unconstrained fine-tuning of flow-matching Vision-Language-Action (VLA) models drives dense parameter overwrites, degrading pre-trained capabilities. We present Conservative Supervised Fine-Tuning (ConSFT), an optimization objective that adapts to target distributions while mitigating catastrophic forgetting, requiring zero prior data or architectural overhead. By dynamically scaling learning signals based on model confidence, ConSFT suppresses excessive gradients from low-confidence samples to prevent disproportionate parameter updates, thereby bounding the intrinsic parameter disruption risk. Inspired by reinforcement learning's trust-region clipping, this formulation establishes a progressive learning dynamic to secure target convergence and prior capability retention, maintaining sparse parameter updates without relying on the parallel reference networks required by explicit regularization. We evaluate ConSFT on the LIBERO and RoboTwin benchmarks across state-of-the-art flow-matching VLAs ($π_0$, $π_{0.5}$, and GR00T-N1.6-3B). The method outperforms vanilla SFT in capability retention by an average absolute margin of over 20\%, matching the efficacy of data-heavy Experience Replay in a prior-data-free regime. Real-world robotic deployments confirm that ConSFT precludes spatial overfitting during downstream adaptation, preserving pre-trained physical skills while acquiring sequential target tasks.

VLA微调机器人持续学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。