让触觉信号与视觉决策高效协同,实现毫秒级响应的智能操作。
TacMamba: A Tactile History Compression Adapter Bridging Fast Reflexes and Slow VLA Reasoning
- 用Mamba结构压缩触觉历史,实现0.45毫秒低延迟处理
- 在按钮识别任务中达成100%成功率,远超纯视觉模型
- 适合需要快速触觉反馈的机器人操控场景
在视觉模糊的操作任务中,触觉反馈往往是唯一可靠的真值来源。然而,触觉数据融合面临时空不匹配难题:触觉感知需高频处理并保留长时记忆(系统1),而视觉策略运行频率较低(系统2)。现有架构难以兼顾:Transformer在高频循环中计算开销过大(>100Hz),LSTM则存在长期记忆遗忘问题。本文提出TacMamba,一种分层架构,实现高频触觉反射与低频视觉规划的对齐。核心贡献包括:(1) 高频触觉接口设计,支持灵活集成;(2) 基于Mamba的触觉历史压缩器,将连续力信号编码为紧凑状态,推理延迟仅0.45毫秒,可无须联合预训练即插即用融合至VLA模型;(3) 触觉引导的双阶段训练策略,利用时间判别进行自监督表征学习,并采用相位均匀采样缓解数据稀疏性。在离散计数和隐式状态切换任务中,TacMamba达到100%成功率,显著优于纯视觉基线pi_0.5,同时严格满足硬实时约束。
原文摘要 · Abstract (English)
In visually ambiguous manipulation such as detecting button click tactile feedback is often the sole source of ground truth. However, fusing tactile data poses a significant challenge due to a spatiotemporal mismatch: tactile perception requires high-frequency processing with long-horizon memory (System 1), whereas visual policies operate at low control frequencies (System 2). Existing architectures struggle to bridge this gap: Transformers are computationally prohibitive for high-frequency loops (>100Hz), while LSTMs suffer from forgetting over extended interaction histories. In this paper, we introduce TacMamba, a hierarchical architecture that aligns high-bandwidth tactile reflexes with low-frequency visual planning. Our approach comprises three core contributions: (1) a custom high-frequency tactile interface designed for flexible integration; (2) a Mamba-based Tactile History Compressor that encodes continuous force history into a compact state with O(1) inference latency (0.45 ms), enabling plug-and-play fusion with VLA models without joint pre-training and (3) a Tactile-Guided Dual-Stage Training strategy that leverages temporal discrimination for self-supervised representation learning and phase-uniform sampling to mitigate data sparsity. Experiments on discrete counting and implicit state switching demonstrate that TacMamba achieves 100% success rates, significantly outperforming the visual-only pi_0.5 baseline, while strictly satisfying hard real-time constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。