让机器人理解执行细节的指令，提升操作精度和可控性。
FineVLA: Fine-Grained Instruction Alignment for Steerable Vision-Language-Action Policies
- 构建细粒度指令数据集，统一10个开源数据集共97万条轨迹
- 混合使用细粒度与粗粒度指令，成功率最高提升23个百分点
- 特别提升对姿态、颜色、方向等细节的控制能力
视觉-语言-动作（VLA）模型需不仅完成机器人任务，还需理解人类对执行方式的指导。但现有数据集通常只提供粗略的目标级语言描述，缺少对主动臂、接近方向、接触区域等执行细节的说明，限制了可操控策略学习和机器人视频理解。本文提出FineVLA，一个用于动作对齐细粒度监督的开源框架，包含：(1) 数据构建工具，整合来自10个开源机器人数据集的972,247条轨迹，构建了经人工验证的47,159条细粒度轨迹数据集FineVLA-Data；(2) 一个独立测试集，含500段视频、11,631个原子事实和1,030个VQA问题；(3) 面向机器人领域的可扩展细粒度标注器；(4) 使用细粒度与原始目标级指令混合训练的可操控VLA策略。实验显示：第一，仅用细粒度指令时，成功率相比纯粗粒度提升1.4至8.1个百分点；第二，两者互补，最优混合比例为1:2至1:1，最优设置下在RoboTwin仿真中成功率达86.8%/82.5%，真实双臂操作中达62.7%/100%（对比纯粗粒度49.9%）；第三，细粒度监督显著提升可操控性，尤其在姿态（+23）、颜色（+18）、接近方向（+18）上获最大提升，而这些正是粗粒度指令无法提供指导的方面。总体而言，应将细粒度语言作为目标级指令的补充，同时说明‘做什么’和‘怎么做’。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models are increasingly expected to not only complete robot tasks, but also follow human instructions about how those tasks should be executed. However, existing robot datasets usually pair trajectories with coarse goal-level language, leaving execution-critical details such as active arm, approach direction, and contact region unspecified. This limits steerable policy learning and robotic video understanding. We introduce FineVLA, an open framework for action-aligned fine-grained VLA supervision. The framework includes: (1) a data construction tool that unifies 972,247 trajectories across 85K tasks from 10 open-source robot datasets and builds FineVLA-Data, a human-verified dataset of 47,159 fine-grained trajectories; (2) a held-out benchmark with 500 videos, 11,631 atomic facts, and 1,030 VQA questions; (3) a robotics-specialized VLM annotator for scalable fine-grained annotation; and (4) a steerable VLA policy trained with controlled mixtures of fine-grained and raw goal-level instructions. Our experiments yield three findings. First, fine-grained supervision does not sacrifice goal-level success: FG-only improves over Raw-only by +1.4 to +8.1 success-rate points across settings. Second, fine-grained and raw instructions are complementary, following a consistent inverted-U trend peaking at FG:Raw = 1:2 to 1:1. The best mixed setting reaches 86.8%/82.5% in RoboTwin simulation and 62.7/100 in real-world dual-arm manipulation (vs. 49.9 Raw-only). Third, fine-grained supervision improves steerable control: the largest real-world gains appear on pose (+23), color (+18), and approach direction (+18)--factors where goal-level instructions provide no guidance. Overall, fine-grained language should augment goal-level instructions: specifying how to execute alongside what to achieve. Project page: https://finevla.xlang.ai/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。