用语言指令控制磁驱动三足软体机器人，实现精准导航
TMR-VLA:Vision-Language-Action Model for Magnetic Motion Control of Tri-leg Silicone-based Soft Robot
- 通过语言+视觉输入，端到端生成电压控制信号
- 在真实环境中实现74%的运动任务成功率
- 适合需要远程操控软体机器人的医疗场景
在体内环境下，磁驱动软体机器人具有无线操作和精确控制的优势，展现出无痛检测与治疗的潜力。我们开发了一种三足磁驱动软体机器人（TMR），其多足设计可实现更灵活的步态和多样化的运动模式。对于由硅胶制成的可重构软体机器人，其导航能力可分解为一系列连续动作，如屈膝、旋转、抬腿、行走等，其运动行为取决于弯曲形状。为连接运动类型描述与底层电压控制，我们提出了TMR-VLA，一种面向三足磁驱动软体机器人的端到端多模态系统，能够执行混合运动类型，有望通过自适应形状实现语言约束下的导航能力。TMR-VLA借鉴了EndoVLA的体内定位能力，融合连续帧图像与自然语言指令作为输入，基于当前观测状态和特定运动类型描述生成低层电压输出。结果表明，TMR-VLA能有效预测施加于TMR的电压如何改变硅基软体机器人的动态行为。TMR-VLA在测试中达到了74%的平均成功率。
原文摘要 · Abstract (English)
In-vivo environments, magnetically actuated soft robots offer advantages such as wireless operation and precise control, showing promising potential for painless detection and therapeutic procedures. We developed a trileg magnetically driven soft robot (TMR) whose multi-legged design enables more flexible gaits and diverse motion patterns. For the silicone made of reconfigurable soft robots, its navigation ability can be separated into sequential motions, namely squatting, rotation, lifting a leg, walking and so on. Its motion and behavior depend on its bending shapes. To bridge motion type description and specific low-level voltage control, we introduced TMR-VLA, an end-to-end multi-modal system for a trileg magnetic soft robot capable of performing hybrid motion types, which is promising for developing a navigation ability by adapting its shape to language-constrained motion types. The TMR-VLA deploys embodied endoluminal localization ability from EndoVLA, and fuses sequential frames and natural language commands as input. Low-level voltage output is generated based on the current observation state and specific motion type description. The result shows the TMR-VLA can predict how the voltage applied to TMR will change the dynamics of a silicon-made soft robot. The TMR-VLA reached a 74% average success rate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。