提出动态触觉-语言框架,让机器人更懂真实世界中的触觉推理。
TacReasoner: A Dynamic Tactile-Language Framework for Interactive Reasoning in Real-World Scenarios

- 设计动态触觉编码器,捕捉随时间变化的触觉信号。
- 构建首个触觉思维链数据集,支持结构化推理。
- 70亿参数模型性能超140亿参数模型,适合轻量级智能系统。
在五种主要人类感官中,触觉对生存最为基础,能感知现实环境中的物理接触与交互。本文聚焦智能系统融合触觉感知时的两大挑战:(i)动态触觉信号建模不足,限制对时序演变特性的推理;(ii)缺乏显式推理机制导致触觉基础模型出现幻觉,造成现实推理不稳定。为此,我们提出TacReasoner,一种面向真实场景交互推理的动态触觉-语言框架。首先,引入动态感知触觉编码器,增强对动态触觉信号的感知与表征能力;更重要的是,构建了首个触觉思维链数据集TouchCoT-10k,支持对触觉输入的结构化推理。基于此,建立DynTac-Bench基准,系统评估动态触觉感知与现实常识推理能力。实验表明,TacReasoner在多个数据集上表现优异,尤其以70亿参数实现超越140亿参数的VTV-LLM模型的多数子任务性能,凸显其在触觉常识推理中的高效性与有效性。
原文摘要 · Abstract (English)
Among the five primary human senses, tactile is arguably the most fundamental to survival, as it enables the perception of physical contact and interaction in real-world environments. In this paper, we explore two key challenges of integrating tactile sensing into intelligent systems for multimodal reasoning: (i) insufficient modeling of dynamic tactile signals, which restricts reasoning over temporally evolving properties, and (ii) hallucination in tactile foundation models caused by the absence of explicit reasoning mechanisms, leading to unstable real-world inference. To address these challenges, we propose TacReasoner, a dynamic tactile-language framework for interactive reasoning in real-world scenarios. First, TacReasoner incorporates a Dynamic-aware Tactile Encoder to enhance the perception and representation of dynamic tactile signals. More importantly, we introduce TouchCoT-10k, the first tactile chain-of-thought dataset for structured reasoning over tactile inputs. Upon it, we establish DynTac-Bench to systematically evaluate dynamic tactile perception and real-world commonsense reasoning. Experimental results demonstrate that TacReasoner achieves competitive performance against state-of-the-art models across multiple datasets. Notably, despite using only 7B parameters, TacReasoner outperforms the 14B VTV-LLM model on most subtasks, highlighting its effectiveness and efficiency in tactile commonsense reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。