让机器人看懂指令并高效执行任务，减少无效视觉处理。
SemanticVLA: Semantic-Aligned Sparsification and Enhancement for Efficient Robotic Manipulation
- 用语义引导的双视觉剪枝器，精准删减冗余图像信息。
- 在仿真和真实场景中，成功率比OpenVLA高21.1%，推理速度提升2.7倍。
- 适合做高效、可解释的智能机器人操控，尤其关注语义对齐。
视觉-语言-动作（VLA）模型在机器人操作中取得进展，但实际部署仍受限于两大问题：1）感知冗余，即无关视觉输入被低效处理；2）指令与视觉表征浅层对齐，导致动作语义基础不牢。本文提出SemanticVLA，一种面向高效机器人操作的语义对齐稀疏化与增强框架。具体包括：1）为在保留语义对齐的同时稀疏冗余感知，设计语义引导双视觉剪枝器（SD-Pruner）：指令驱动剪枝器（ID-Pruner）从SigLIP中提取全局动作线索与局部语义锚点；空间聚合剪枝器（SA-Pruner）将几何丰富特征压缩为任务自适应的令牌，基于DINOv2实现。2）为利用稀疏特征并融合语义与空间几何，引入语义互补层次融合器（SH-Fuser），在SigLIP与DINOv2间融合密集图块与稀疏令牌，生成一致表征。3）为提升感知到动作的转换效率，采用语义条件动作耦合器（SA-Coupler），替代传统观测到自由度映射，实现更高效且可解释的行为建模。在仿真与真实任务上大量实验表明，SemanticVLA在性能与效率上均达到新SOTA。其在LIBERO基准上成功率较OpenVLA提升21.1%，训练成本与推理延迟分别降低3.0倍和2.7倍。代码已开源，详见https://github.com/JiuTian-VL/SemanticVLA。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models have advanced in robotic manipulation, yet practical deployment remains hindered by two key limitations: 1) perceptual redundancy, where irrelevant visual inputs are processed inefficiently, and 2) superficial instruction-vision alignment, which hampers semantic grounding of actions. In this paper, we propose SemanticVLA, a novel VLA framework that performs Semantic-Aligned Sparsification and Enhancement for Efficient Robotic Manipulation. Specifically: 1) To sparsify redundant perception while preserving semantic alignment, Semantic-guided Dual Visual Pruner (SD-Pruner) performs: Instruction-driven Pruner (ID-Pruner) extracts global action cues and local semantic anchors in SigLIP; Spatial-aggregation Pruner (SA-Pruner) compacts geometry-rich features into task-adaptive tokens in DINOv2. 2) To exploit sparsified features and integrate semantics with spatial geometry, Semantic-complementary Hierarchical Fuser (SH-Fuser) fuses dense patches and sparse tokens across SigLIP and DINOv2 for coherent representation. 3) To enhance the transformation from perception to action, Semantic-conditioned Action Coupler (SA-Coupler) replaces the conventional observation-to-DoF approach, yielding more efficient and interpretable behavior modeling for manipulation tasks. Extensive experiments on simulation and real-world tasks show that SemanticVLA sets a new SOTA in both performance and efficiency. SemanticVLA surpasses OpenVLA on LIBERO benchmark by 21.1% in success rate, while reducing training cost and inference latency by 3.0-fold and 2.7-fold.SemanticVLA is open-sourced and publicly available at https://github.com/JiuTian-VL/SemanticVLA
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。