arXiv:2605.00475cs.ROcs.CV2026-05

解决双手精细操作中视觉定位漂移问题,实现低延迟稳定控制。

MSACT: Multistage Spatial Alignment for Stable Low-Latency Fine Manipulation

论文配图:MSACT: Multistage Spatial Alignment for Stable Low-Latency Fine Manipulation
图 1 · 摘自论文原文
  • 分阶段空间注意力机制提取稳定2D注意力点,辅助动作预测。
  • 通过未来帧对齐损失抑制定位漂移,提升视觉-动作映射稳定性。
  • 适合资源受限的实时精细操作场景,尤其适用于数据有限时。

真实世界中的精细操作,尤其是双手操作,通常需要低延迟控制和稳定的视觉定位。然而,大规模数据收集成本高,少量示范易导致定位漂移。现有方法各有权衡:如ACT等动作分块策略虽实现低延迟与数据高效,但依赖密集视觉特征且缺乏显式空间一致性;扩散策略等生成方法表达能力强但采样延迟高;基于视觉语言或体素的方法虽增强泛化性与几何对齐,但计算开销大、系统复杂。本文提出多阶段空间注意力模块,通过预训练ResNet视觉先验,在ACT框架上提取任务相关的2D注意力点作为局部空间模态,并引入自监督时间对齐损失,使预测注意力序列与未来帧视觉特征对齐,从而在无需关键点标注的情况下抑制漂移,提升有限数据下的视觉-动作映射稳定性。在ALOHA双臂平台上进行的仿真与真实实验表明,该方法在保持低延迟推理的同时,显著提升定位稳定性与任务成功率,且对视觉干扰具有更强鲁棒性。

原文摘要 · Abstract (English)

Real-world fine manipulation, particularly in bimanual manipulation, typically requires low-latency control and stable visual localization, while collecting large-scale data is costly and limited demonstrations may lead to localization drift. Existing approaches make different trade-offs: action-chunking policies such as ACT enable low-latency execution and data efficiency but rely on dense visual features without explicit spatial consistency, generative methods such as Diffusion Policy improve expressiveness but can incur iterative sampling latency, vision-language-action and voxel-based methods enhance generalization and geometric grounding but require higher computational cost and system complexity. We introduce a multistage spatial attention module that extracts stable 2D attention points and jointly predicts future attention sequences with a temporal alignment loss. Built upon ACT with a pretrained ResNet visual prior, a multistage attention module extracts task-relevant 2D attention points as a local spatial modality for action prediction. To maintain consistent object tracking, we introduce a self-supervised objective that aligns predicted attention sequences with visual features from future frames, suppressing drift without keypoint annotations and improving stability of the vision-to-action mapping under limited data. Experiments on simulated and real-world fine manipulation tasks, conducted on the ALOHA bimanual platform, evaluate task success, attention drift, inference latency, and robustness to visual disturbances. Results indicate improvements in localization stability and task performance while maintaining low-latency inference under the tested conditions.

精细操作空间对齐低延迟

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。