用物理仿真+视觉语言模型,让机器人更真实地学习切菜。
CulinaryCut-VLAP: A Vision-Language-Action-Physics Framework for Food Cutting via a Force-Aware Material Point Method
- 结合视觉语言动作与物理模拟,用材料点法建模刀具切割过程。
- 模拟中实现力与应力的稳定追踪,支持大变形和拓扑变化场景。
- 提供含力矩、姿态标签的数据集,适合研究可泛化的切菜机器人。
食品切割是视觉与机器人操作交叉的实用但未充分探索的任务。由于刀具与柔性材料间的交互高度非线性,常伴随大变形、频繁接触及拓扑变化,导致大规模数据采集难以稳定安全进行。为此,我们提出一个统一框架,将视觉-语言-动作(VLA)数据集与基于材料点法(MPM)的物理逼真切割模拟器相结合。模拟器采用MLS-MPM作为核心计算方法,在保持旋转与剪切响应的同时,有效降低数值耗散与能量漂移,即使在拓扑变化切割中也能稳定运行。通过粒子与网格间的冲量交换估算力与应力分布,实现瞬态接触力与能量传递的精确追踪。同时,我们构建了一个基准数据集,包含多样化的切割轨迹、多视角视觉观测、细粒度语言指令,以及力-扭矩和工具-位姿标签,为训练提供物理一致信号。该框架实现了学习-评估闭环,尊重切割核心物理规律,建立了安全、可复现、可扩展的变形体操作研究基础。
原文摘要 · Abstract (English)
Food cutting is a highly practical yet underexplored application at the intersection of vision and robotic manipulation. The task remains challenging because interactions between the knife and deformable materials are highly nonlinear and often entail large deformations, frequent contact, and topological change, which in turn hinder stable and safe large-scale data collection. To address these challenges, we propose a unified framework that couples a vision-language-action (VLA) dataset with a physically realistic cutting simulator built on the material point method (MPM). Our simulator adopts MLS-MPM as its computational core, reducing numerical dissipation and energy drift while preserving rotational and shear responses even under topology-changing cuts. During cutting, forces and stress distributions are estimated from impulse exchanges between particles and the grid, enabling stable tracking of transient contact forces and energy transfer. We also provide a benchmark dataset that integrates diverse cutting trajectories, multi-view visual observations, and fine-grained language instructions, together with force--torque and tool--pose labels to provide physically consistent training signals. These components realize a learning--evaluation loop that respects the core physics of cutting and establishes a safe, reproducible, and scalable foundation for advancing VLA models in deformable object manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。