构建室内物体材质数据集,助力机器人视觉精准识别材质。
MatPredict: a dataset and benchmark for learning material properties of diverse indoor objects
- 融合Replica与MatSynth数据,生成18类物体、14种材质的高仿真图像。
- 在不同光照与视角下渲染,支持材质属性的多条件测试。
- 适合做机器人感知、材质识别与仿真训练的研究者使用。
从相机图像中推断材料属性可增强室内环境中复杂物体的识别能力,对消费级机器人应用具有重要意义。为此,我们引入了MatPredict数据集,将Replica数据集的高质量合成物体与MatSynth数据集的材料属性类别相结合,创建具有多样化材料属性的物体。我们选取特定前景物体的3D网格,并以不同材料属性进行渲染。总共生成了18种常见物体和14种不同材料。我们展示了在光照和相机位置上的变化性。随后,我们提供了一个基准,用于通过这些场景中的扰动模型从视觉图像中推断材料属性,讨论了涉及的具体神经网络模型及其在不同图像比较指标下的性能表现。通过准确模拟不同材料与光的交互,可以提升真实性,这对通过大规模仿真有效训练模型至关重要。本研究旨在革新消费机器人中的感知能力。数据集可通过https://huggingface.co/datasets/UMTRI/MatPredict获取,代码可在https://github.com/arpan-kusari/MatPredict获得。
原文摘要 · Abstract (English)
Determining material properties from camera images can expand the ability to identify complex objects in indoor environments, which is valuable for consumer robotics applications. To support this, we introduce MatPredict, a dataset that combines the high-quality synthetic objects from Replica dataset with MatSynth dataset's material properties classes - to create objects with diverse material properties. We select 3D meshes of specific foreground objects and render them with different material properties. In total, we generate \textbf{18} commonly occurring objects with \textbf{14} different materials. We showcase how we provide variability in terms of lighting and camera placement for these objects. Next, we provide a benchmark for inferring material properties from visual images using these perturbed models in the scene, discussing the specific neural network models involved and their performance based on different image comparison metrics. By accurately simulating light interactions with different materials, we can enhance realism, which is crucial for training models effectively through large-scale simulations. This research aims to revolutionize perception in consumer robotics. The dataset is provided \href{https://huggingface.co/datasets/UMTRI/MatPredict}{here} and the code is provided \href{https://github.com/arpan-kusari/MatPredict}{here}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。