用麦克风指尖捕捉触觉音频，提升按按钮时的力度控制。
You're Pushing My Buttons: Instrumented Learning of Gentle Button Presses
- 用指尖麦克风采集接触声音，训练音频编码器识别触碰事件。
- 训练中引入传感器信号，使按压动作力道降低30%以上。
- 部署时仅需视觉与音频，适合机器人精细操作任务。
仅依靠摄像头和本体感知学习接触密集型操作非常困难，因为接触事件只能部分观测。我们测试了训练阶段仪器化（即物体传感）是否能提升策略性能，且不产生部署依赖。以按按钮为实验场景，使用麦克风指尖捕获接触相关音频。利用仪器化按钮状态信号作为特权监督，微调音频编码器成为接触事件检测器。将所得表征结合模仿学习，采用三种策略，使推理阶段仅使用视觉与音频。各类方法的按压成功率相近，但基于仪器化的音频表征始终能有效降低接触力。结果表明，仪器化可作为学习接触密集型操作策略的实用训练辅助目标。
原文摘要 · Abstract (English)
Learning contact-rich manipulation is difficult from cameras and proprioception alone because contact events are only partially observed. We test whether training-time instrumentation, i.e., object sensorisation, can improve policy performance without creating deployment-time dependencies. Specifically, we study button pressing as a testbed and use a microphone fingertip to capture contact-relevant audio. We use an instrumented button-state signal as privileged supervision to fine-tune an audio encoder into a contact event detector. We combine the resulting representation with imitation learning using three strategies, such that the policy only uses vision and audio during inference. Button press success rates are similar across methods, but instrumentation-guided audio representations consistently reduce contact force. These results support instrumentation as a practical training-time auxiliary objective for learning contact-rich manipulation policies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。