arXiv:2607.10706cs.ROcs.AI2026-07

将3D机械臂操作转为像素分类,实现高精度、快速推理的闭环控制。

Action Map Policy: Learning 3D Closed-loop Manipulation via Pixel Classification

论文配图:Action Map Policy: Learning 3D Closed-loop Manipulation via Pixel Classification
图 1 · 摘自论文原文
  • 用像素位置作为动作类别,解决高维动作空间的维度爆炸问题。
  • 在多个任务上成功率达85%以上,推理速度比扩散模型快10倍。
  • 适合需要精细视觉反馈和实时响应的机器人操控场景。

动作空间是机器人学习中的主要挑战,因其维度高、时间跨度长且常存在多模态最优解。现有方法在动作表示与损失函数间常有权衡。本文提出动作图策略(AMP),将3D闭环操纵策略学习转化为图像空间中的分类问题。尽管分类在生成式语言模型中有效,但直接离散化高维连续动作会导致词表过大。我们的核心思想是将3D动作投影到相机图像平面,以每个像素位置作为离散类别,从而控制维度并保留多模态性。该方法支持毫米级精度的高维动作预测,无需超大词表,同时保持细粒度像素级视觉信号。此外,可在单次前向传播中预测完整动作块,避免复杂的噪声调度与迭代去噪,显著提升推理速度。在多种操纵任务上的实验表明,AMP优于强基线,成功率更高,推理更快,空间推理能力更强。

原文摘要 · Abstract (English)

The action space poses a major challenge in robot learning, since it is often high-dimensional, can span long time horizons, and frequently admits multi-modal optimal solutions. A good choice of action representation and loss function can help to address these concerns, but there are often trade offs. We propose Action Map Policy (AMP), which casts 3D closed-loop manipulation policy learning as a classification problem in image space. While classification has been an effective formulation in generative language models, applying it to robot action learning is difficult because naively discretizing high-dimensional continuous actions explodes the token vocabulary. Our key idea is to project 3D actions onto the camera image planes and treat each pixel location as a discrete class, thus controlling dimensionality while retaining multi-modality. This method supports millimeter-level precision for high-dimensional actions without requiring a prohibitively large vocabulary, while preserving fine-grained pixel-wise visual signals. Furthermore, it can predict the entire action chunk in a single forward pass, avoiding complex noise scheduling and iterative denoising while achieving substantially faster inference than diffusion policies. Experiments on various manipulation tasks show that AMP outperforms strong baselines, achieving higher success rates, faster inference, and enhanced spatial reasoning.

机器人操控动作分类图像投影

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。