用频域分层建模动作，提升机器人抓取的精度与效率
FreqPolicy: Frequency Autoregressive Visuomotor Policy with Continuous Tokens
- 将动作分解为频域中的高低频分量，分层生成
- 在2D/3D任务中平均成功率超基线17.3%，推理速度更快
- 适合需要高精度动作控制的复杂机器人操作场景
学习有效的视觉-运动策略以实现机器人操作极具挑战性，因需生成精确动作的同时保持计算高效。现有方法受限于动作表示方式和基础网络结构。我们发现，在频域中表示动作能更有效捕捉运动的结构性：低频成分反映全局运动模式，高频成分编码局部精细细节。且不同复杂度的任务对各频段建模精度要求不同。受此启发，我们提出一种新型视觉-运动策略学习范式，逐步建模分层频域成分。为进一步提升精度，引入连续潜变量表示，保持动作空间的平滑性和连续性。在多个2D和3D机器人操作基准上进行的大量实验表明，该方法在准确率和效率上均优于现有方法，展示了基于频域自回归框架与连续令牌的通用机器人操作潜力。代码已公开于 https://github.com/4DVLab/Freqpolicy。
原文摘要 · Abstract (English)
Learning effective visuomotor policies for robotic manipulation is challenging, as it requires generating precise actions while maintaining computational efficiency. Existing methods remain unsatisfactory due to inherent limitations in the essential action representation and the basic network architectures. We observe that representing actions in the frequency domain captures the structured nature of motion more effectively: low-frequency components reflect global movement patterns, while high-frequency components encode fine local details. Additionally, robotic manipulation tasks of varying complexity demand different levels of modeling precision across these frequency bands. Motivated by this, we propose a novel paradigm for visuomotor policy learning that progressively models hierarchical frequency components. To further enhance precision, we introduce continuous latent representations that maintain smoothness and continuity in the action space. Extensive experiments across diverse 2D and 3D robotic manipulation benchmarks demonstrate that our approach outperforms existing methods in both accuracy and efficiency, showcasing the potential of a frequency-domain autoregressive framework with continuous tokens for generalized robotic manipulation.Code is available at https://github.com/4DVLab/Freqpolicy
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。