用3D场景推理生成机器人动作，不依赖传统策略学习。
Avi: Action from Volumetric Inference
- 基于3D点云和语言理解，通过几何变换计算动作
- 无需训练历史动作，直接生成下一帧点云并推导动作
- 对遮挡、视角变化鲁棒，适合复杂环境中的通用机器人
我们提出Avi，一种新型的3D视觉-语言-动作（VLA）架构，将机器人动作生成重构为3D感知与空间推理问题，而非低层策略学习。现有VLA模型主要处理2D视觉输入，并在特定任务上端到端训练动作策略，而Avi利用3D点云与语言引导的场景理解，通过经典几何变换计算动作。尤为关键的是，Avi不依赖历史动作标记，而是基于3D多模态大语言模型生成下一帧点云，并显式计算动作。该方法使行为具有可泛化性，对遮挡、相机位姿变化和视点改变均保持鲁棒。通过将机器人决策视为对3D表示的结构化推理任务，Avi在高层语言指令与底层执行之间搭建桥梁，无需透明策略学习。初步结果表明，3D视觉-语言推理可成为可扩展、鲁棒机器人系统的基础。详见 https://avi-3drobot.github.io/。
原文摘要 · Abstract (English)
We propose Avi, a novel 3D Vision-Language-Action (VLA) architecture that reframes robotic action generation as a problem of 3D perception and spatial reasoning, rather than low-level policy learning. While existing VLA models primarily operate on 2D visual inputs and are trained end-to-end on task-specific action policies, Avi leverages 3D point clouds and language-grounded scene understanding to compute actions through classical geometric transformations. Most notably, Avi does not train on previous action tokens, rather, we build upon a 3D Multi-modal Large Language Model (MLLM) to generate the next point cloud and explicitly calculate the actions through classical transformations. This approach enables generalizable behaviors that are robust to occlusions, camera pose variations, and changes in viewpoint. By treating the robotic decision-making process as a structured reasoning task over 3D representations, Avi bridges the gap between high-level language instructions and low-level actuation without requiring opaque policy learning. Our preliminary results highlight the potential of 3D vision-language reasoning as a foundation for scalable, robust robotic systems. Check it out at https://avi-3drobot.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。