arXiv:2607.09218cs.ROcs.AI2026-07

用触觉与视觉引导,实现多接触点的机械臂精准控制。

TACTIC: Tactile and Vision Conditioned Contact-Centric Control for Whole-Arm Manipulation

论文配图:TACTIC: Tactile and Vision Conditioned Contact-Centric Control for Whole-Arm Manipulation
图 1 · 摘自论文原文
  • 基于触觉、视觉和接触状态预测,构建接触中心型控制器。
  • 在仿真中优于现有模型方法,在真实机器人上完成三项复杂任务。
  • 适合需要多点接触交互的工业或服务机器人场景。

全臂操作涉及机器人在任务中与环境直接接触,通过多个关节分布接触力,形成、滑动和分离接触。这打破了多数学习型操作方法的隐含假设:机械臂构型紧密耦合运动与接触力,接触状态受遮挡部分可观测,且纯学习的轨迹在分布偏移下可能物理不一致,因多链接接触配置在数据中稀疏。为此,我们提出TACTIC(触觉与视觉条件下的接触中心控制),一种用于全臂操作的滚动时域控制器。TACTIC采用接触中心型混合预测模型,融合RGB-D图像、分布式触觉传感和紧凑2D邻近表示。该模型通过接触雅可比矩阵将学习的条件动作隐空间动力学与解析运动学耦合,可推演未来接触配置与相互作用力。TACTIC将这些推演集成至采样式模型预测控制(MPC)规划器中,采用接触感知的动作采样:基于接触雅可比的投影引导采样动作序列朝向力调节方向,目标函数则在预测邻近度与交互力之间权衡任务进展与全臂力调控。我们在仿真中对比了最先进的模型基与无模型方法,并进行消融实验以隔离各设计选择的贡献。TACTIC表现持续领先。进一步在配备分布式触觉传感的真实机器人上验证,完成三项需多接触轨迹的任务:翻转并重定位一个假人,以及在三维动态迷宫中达成目标。网站:https://emprise.cs.cornell.edu/tactic

原文摘要 · Abstract (English)

Whole-arm manipulation involves direct contact with the environment while the robot completes a task by distributing contact across multiple links as contacts form, slide, and break. This setting breaks common implicit assumptions in many learning-based manipulation pipelines: arm configuration tightly couples motion and contact forces, contact state is partially observed under occlusion, and purely learned rollouts can become physically inconsistent under distribution shift because many multi-link contact configurations are sparsely represented in the data. To address this, we propose TACTIC (Tactile and Vision Conditioned Contact-Centric Control), a receding-horizon controller for whole-arm manipulation. TACTIC uses a contact-centric hybrid predictive model that combines RGB-D, distributed tactile sensing, and a compact 2D proximity representation. The model couples a learned, action-conditioned latent dynamics model with analytical kinematics through contact Jacobians, enabling rollouts of future contact configurations and interaction forces. TACTIC integrates these rollouts into a sampling-based MPC planner with contact-aware action sampling: contact Jacobian-based projections steer sampled action sequences toward force-modulating directions, and objectives defined over predicted proximity and interaction forces trade task progress against whole-arm force regulation. We evaluate TACTIC in simulation against state-of-the-art model-based and model-free methods, and perform ablations that isolate the contribution of each design choice. TACTIC consistently outperforms other methods. We further demonstrate real-world performance on a robot with distributed tactile sensing across three whole-arm manipulation tasks that require multi-contact trajectories: turning over and repositioning a manikin, and goal-reaching in a 3D dynamic maze. Website: https://emprise.cs.cornell.edu/tactic

机械臂控制触觉感知多接触交互模型预测控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。