用视觉和机器人状态模拟力觉,让无传感器机械臂也能精准操作。
FD-VLA: Force-Distilled Vision-Language-Action Model for Contact-Rich Manipulation
- 通过可学习查询令牌,从视觉和状态信息中蒸馏出力觉信号。
- 物理实验显示,模拟力觉比真实传感器数据表现更好。
- 适合无力传感器的低成本机械臂,提升接触任务的鲁棒性。
力感知是视觉-语言-动作(VLA)框架在高接触任务中实现精细感知与灵巧操作的关键模态。本文提出力蒸馏视觉-语言-动作模型(FD-VLA),无需物理力传感器即可融入力觉信息。核心是力蒸馏模块(FDM),它将一个基于视觉观测和机器人状态的可学习查询令牌映射为与真实力信号潜在表示对齐的预测力令牌。推理时,该蒸馏力令牌被注入预训练VLM,实现力觉感知的同时保持视觉-语言语义完整性。该设计带来双重优势:一是支持广泛无成本、抗损的机器人部署,降低硬件复杂度;二是在VLM前引入力-视觉-状态融合先验,增强跨模态对齐与接触场景下的感知-动作鲁棒性。令人意外的是,物理实验证明,蒸馏力令牌的表现优于直接传感器测量和其他基线方法,验证了该力蒸馏VLA方法的有效性。
原文摘要 · Abstract (English)
Force sensing is a crucial modality for Vision-Language-Action (VLA) frameworks, as it enables fine-grained perception and dexterous manipulation in contact-rich tasks. We present Force-Distilled VLA (FD-VLA), a novel framework that integrates force awareness into contact-rich manipulation without relying on physical force sensors. The core of our approach is a Force Distillation Module (FDM), which distills force by mapping a learnable query token, conditioned on visual observations and robot states, into a predicted force token aligned with the latent representation of actual force signals. During inference, this distilled force token is injected into the pretrained VLM, enabling force-aware reasoning while preserving the integrity of its vision-language semantics. This design provides two key benefits: first, it allows practical deployment across a wide range of robots that lack expensive or fragile force-torque sensors, thereby reducing hardware cost and complexity; second, the FDM introduces an additional force-vision-state fusion prior to the VLM, which improves cross-modal alignment and enhances perception-action robustness in contact-rich scenarios. Surprisingly, our physical experiments show that the distilled force token outperforms direct sensor force measurements as well as other baselines, which highlights the effectiveness of this force-distilled VLA approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。