用自然语言在陌生室内环境完成移动抓取，系统全程运行在消费级硬件上。
AnywhereVLA: Language-Conditioned Exploration and Mobile Manipulation
- 基于语言指令构建任务图，结合传统SLAM与语义地图进行探索
- 在多房间场景中实现46%的任务成功率，实时运行于嵌入式设备
- 融合几何导航可靠性与语言操控泛化能力，适合移动机器人应用
我们提出AnywhereVLA，一个用于在未见过的、不可预测的室内环境中实现自然语言控制的抓取与放置任务的模块化移动操作框架。用户输入文本提示后，系统将其解析为结构化任务图，指导基于激光雷达和摄像头的经典SLAM、度量语义映射以及任务感知的前沿探索策略。随后，规划器生成考虑可见性与可达性的预抓取基座姿态。交互阶段，通过在TheRobotStudio的SO-101数据集上微调的紧凑型SmolVLA操作头，将局部视觉上下文与子目标转化为抓取和放置建议。整个系统完全在机载硬件上运行：Jetson Orin NX负责感知与视觉语言动作（VLA），Intel NUC处理SLAM、探索与控制，维持实时性能。我们在多房间实验室静态场景及正常人类活动条件下进行了评估，系统整体任务成功率达46%，同时保持嵌入式计算平台上的高吞吐量。通过结合经典系统栈与微调后的视觉语言动作模型，该系统兼具几何导航的可靠性与语言驱动操作的灵活性与泛化能力。
原文摘要 · Abstract (English)
We address natural language pick-and-place in unseen, unpredictable indoor environments with AnywhereVLA, a modular framework for mobile manipulation. A user text prompt serves as an entry point and is parsed into a structured task graph that conditions classical SLAM with LiDAR and cameras, metric semantic mapping, and a task-aware frontier exploration policy. An approach planner then selects visibility and reachability aware pre grasp base poses. For interaction, a compact SmolVLA manipulation head is fine tuned on platform pick and place trajectories for the SO-101 by TheRobotStudio, grounding local visual context and sub-goals into grasp and place proposals. The full system runs fully onboard on consumer-level hardware, with Jetson Orin NX for perception and VLA and an Intel NUC for SLAM, exploration, and control, sustaining real-time operation. We evaluated AnywhereVLA in a multi-room lab under static scenes and normal human motion. In this setting, the system achieves a $46\%$ overall task success rate while maintaining throughput on embedded compute. By combining a classical stack with a fine-tuned VLA manipulation, the system inherits the reliability of geometry-based navigation with the agility and task generalization of language-conditioned manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。