用触觉反馈和3D高斯点云，实现水果无损抓取与损伤量化评估
DexFruit: Dexterous Manipulation and Gaussian Splatting Inspection of Fruit
- 融合光学触觉感知的扩散策略实现自适应柔抓
- 抓取成功率92%，视觉伤痕减少最多20%
- 支持任意2D分割模型的3D损伤建模，适合农业机器人研发
DexFruit 是一个机器人操作框架，可实现对易损水果的温和自主抓取及精确损伤评估。由于多数水果易受压伤，需人工小心采摘。本文利用光学触觉传感，证明了基于触觉信息的扩散策略在草莓、番茄和黑莓三种水果上均能显著降低损伤并提升抓取成功率。我们提出 FruitSplat 技术，通过 3D 高斯点云（3DGS）构建高分辨率三维损伤表示，将 2D 草莓掩码和 2D 损伤分割掩码映射至 3DGS。该方法无需昂贵设备，具备模块化与通用性，兼容任意 2D 模型。在超过 630 次试验中，系统实现 92% 的抓取成功率，视觉伤痕减少最高达 20%，在挑战性水果上抓取成功率提升最高达 31%，优于基线方法。
原文摘要 · Abstract (English)
DexFruit is a robotic manipulation framework that enables gentle, autonomous handling of fragile fruit and precise evaluation of damage. Many fruits are fragile and prone to bruising, thus requiring humans to manually harvest them with care. In this work, we demonstrate by using optical tactile sensing, autonomous manipulation of fruit with minimal damage can be achieved. We show that our tactile informed diffusion policies outperform baselines in both reduced bruising and pick-and-place success rate across three fruits: strawberries, tomatoes, and blackberries. In addition, we introduce FruitSplat, a novel technique to represent and quantify visual damage in high-resolution 3D representation via 3D Gaussian Splatting (3DGS). Existing metrics for measuring damage lack quantitative rigor or require expensive equipment. With FruitSplat, we distill a 2D strawberry mask as well as a 2D bruise segmentation mask into the 3DGS representation. Furthermore, this representation is modular and general, compatible with any relevant 2D model. Overall, we demonstrate a 92% grasping policy success rate, up to a 20% reduction in visual bruising, and up to an 31% improvement in grasp success rate on challenging fruit compared to our baselines across our three tested fruits. We rigorously evaluate this result with over 630 trials. Please checkout our website at https://dex-fruit.github.io .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。