arXiv:2412.02119cs.CVcs.LG2024-12被引 3

仅用视频分析颗粒物大小密度,无需传感器和人工标注。

Understanding Particles From Video: Property Estimation of Granular Materials via Visuo-Haptic Learning

  • 基于视觉触觉学习框架,从颗粒互动视频中推断属性。
  • 模型能准确估计颗粒相对大小与密度分布,误差低于10%。
  • 适合农业、工业等需快速评估颗粒材料的场景。

颗粒材料在日常生活和工业中普遍存在,其属性理解至关重要,尤其在农业与制造业。现有方法依赖专用测量设备,且需大量人力处理大量颗粒。本文提出一种从颗粒材料互动视频中估计相对粒径与密度的方法。该方法基于受接触模型启发的视觉-触觉学习框架,揭示了颗粒属性与探针拖动过程中视觉-触觉数据间的强相关性。训练后,网络可将视觉模态有效映射至触觉信号,并在潜在嵌入中隐式表征颗粒属性的相对分布。因此,仅通过训练后的编码器与视觉信息即可分析颗粒属性,无需额外传感模态或人工标注。所提颗粒属性估计算法经多组对比与消融实验验证,泛化能力亦经测试,真实海滩场景应用已成功演示。实验视频见:https://sites.google.com/view/gmwork/vhlearning。

原文摘要 · Abstract (English)

Granular materials (GMs) are ubiquitous in daily life. Understanding their properties is also important, especially in agriculture and industry. However, existing works require dedicated measurement equipment and also need large human efforts to handle a large number of particles. In this paper, we introduce a method for estimating the relative values of particle size and density from the video of the interaction with GMs. It is trained on a visuo-haptic learning framework inspired by a contact model, which reveals the strong correlation between GM properties and the visual-haptic data during the probe-dragging in the GMs. After training, the network can map the visual modality well to the haptic signal and implicitly characterize the relative distribution of particle properties in its latent embeddings, as interpreted in that contact model. Therefore, we can analyze GM properties using the trained encoder, and only visual information is needed without extra sensory modalities and human efforts for labeling. The presented GM property estimator has been extensively validated via comparison and ablation experiments. The generalization capability has also been evaluated and a real-world application on the beach is also demonstrated. Experiment videos are available at \url{https://sites.google.com/view/gmwork/vhlearning} .

颗粒材料视觉学习属性估计无监督

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。