融合深度对齐与GRU预测,实现高精度低延迟的视觉伺服控制。
Hybrid Vision Servoing with Depp Alignment and GRU-Based Occlusion Recovery
- 分阶段融合全局匹配与浅层深度特征优化定位
- 在90%遮挡下仍保持<2像素误差,支持30Hz实时控制
- 适合需鲁棒视觉跟踪的工业机器人操作场景
基于视觉的控制系统(如基于图像的视觉伺服,IBVS)在机器人精确操作中被广泛研究。然而,在部分或完全遮挡情况下保持目标稳定跟踪仍是持久挑战。传统方法如Lucas-Kanade(LK)虽轻量但易受遮挡和漂移影响,而深度学习方法通常依赖连续可见性且计算开销大。为此,我们提出一种混合视觉跟踪框架,将先进感知与实时伺服控制结合:首先通过快速全局模板匹配限制姿态搜索区域;接着在VGG浅层特征上运行深度特征LK模块,实现亚像素级对齐(<2像素);然后使用轻量残差回归器修正因纹理退化或局部遮挡引起的局部偏差;当视觉置信度低于阈值时,基于GRU的预测器从近期运动历史中无缝外推姿态更新。关键的是,整个流程最终输出的平移、旋转和尺度增量直接作为30Hz IBVS控制环的输入信号。在手持视频序列(最高90%遮挡)上的评估表明,系统可维持<2像素的跟踪误差,证明了其在真实机器人视觉应用中所需的鲁棒性与低延迟精度。
原文摘要 · Abstract (English)
Vision-based control systems, such as image-based visual servoing (IBVS), have been extensively explored for precise robot manipulation. A persistent challenge, however, is maintaining robust target tracking under partial or full occlusions. Classical methods like Lucas-Kanade (LK) offer lightweight tracking but are fragile to occlusion and drift, while deep learning-based approaches often require continuous visibility and intensive computation. To address these gaps, we propose a hybrid visual tracking framework that bridges advanced perception with real-time servo control. First, a fast global template matcher constrains the pose search region; next, a deep-feature Lucas-Kanade module operating on early VGG layers refines alignment to sub-pixel accuracy (<2px); then, a lightweight residual regressor corrects local misalignments caused by texture degradation or partial occlusion. When visual confidence falls below a threshold, a GRU-based predictor seamlessly extrapolates pose updates from recent motion history. Crucially, the pipeline's final outputs-translation, rotation, and scale deltas-are packaged as direct control signals for 30Hz image-based servo loops. Evaluated on handheld video sequences with up to 90% occlusion, our system sustains under 2px tracking error, demonstrating the robustness and low-latency precision essential for reliable real-world robot vision applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。