按动作组分别评估误差，比总均方误差更可靠。
Per-Group Error, Not Total MSE: Fine-Tuning Vision-Language-Action Models for 11-DoF Mobile Manipulation
- 按关节组分项评估误差，避免易预测组掩盖难预测组
- 实机测试中总误差最低的模型表现反而最差
- 适合有异构动作空间的机器人系统调优
为异构关节空间的移动操作机器人微调视觉-语言-动作（VLA）模型时，总均方误差最低的检查点未必在真实机器人上表现最好。这是因为将机械臂、夹爪、头部和轮式底盘等不同关节组合并为单一指标，导致易预测的关节会掩盖仍出错的关节。本文在11自由度丰田HSR机器人上微调450M的SmolVLA（仅动作专家），并与更强的3.3B预训练基线$π_{0.5}$对比。分组分析显示：SmolVLA的移动底盘收敛最慢，成为性能瓶颈；而对$π_{0.5}$仅微调动作头（冻结主干）时，总均方误差虽低于基线，但机械臂精度下降。60次真实机器人实验（每模型20次）表明，$π_{0.5}$ 80k（4.0/4）显著优于两个微调版本（专家仅3k：3.75/4；HSR-SmolVLA：3.5/4；Mann-Whitney $p \leq 0.010$），尽管后者总误差更低。性能差异与离线机械臂组误差最一致，而非总误差或底盘组误差。结论：对于异构动作空间的机器人，应以每组误差作为检查点选择依据，而非总均方误差。
原文摘要 · Abstract (English)
Fine-tuning Vision-Language-Action (VLA) models for mobile manipulators with heterogeneous joint spaces can produce a counterintuitive result: the checkpoint with the lowest aggregate MSE is not the one that performs best on the real robot. We argue this is a predictable consequence of collapsing heterogeneous joint groups (arm, gripper, head, wheeled base) into a single metric, where easy-to-predict joints can mask joints that still fail. We fine-tune SmolVLA (450M, action-expert only) on the 11-DoF Toyota HSR and compare it against $π_{0.5}$ (3.3B), a stronger pretrained baseline. Per-group analysis exposes two patterns: in SmolVLA, the mobile base converges slowest and limits overall performance. In expert-only fine-tuning of $π_{0.5}$ (training only the action head, backbone frozen), total MSE drops below the baseline but arm accuracy degrades. On 60 real-robot trials (20 per model), $π_{0.5}$ 80k (4.0/4) significantly outperforms both fine-tuned variants (expert-only 3k: 3.75/4; HSR-SmolVLA: 3.5/4; Mann-Whitney $p \leq 0.010$), despite expert-only 3k having the lowest total MSE. This separation is most consistent with the offline arm-group error, not total MSE or base-group error. We conclude that per-group error is a more reliable signal than total MSE for checkpoint selection on robots with heterogeneous action spaces. Code: https://github.com/paumontagut/per-group-mse-vla
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。