用大模型从点云推断物体材质，无需训练即可准确判断材质与语义关系。
LLM-Guided Material Inference for 3D Point Clouds
- 分两阶段推理：先识别物体语义，再根据语义匹配合理材质。
- 在1000个点云上实现高可信的语义与材质合理性，零样本无训练。
- 适合3D内容生成、虚拟场景构建等需要真实材质的应用。
现有3D形状数据集和模型主要关注几何结构，忽视决定物体外观的材质属性。本文提出一种基于大语言模型（LLM）的两阶段方法，直接从带有粗分割的3D点云中推断材质组成。核心思想是将物体身份识别与材质判断分离处理：第一阶段由LLM预测物体语义；第二阶段在已知语义条件下，为每个几何片段分配合理的材质。两个阶段均采用零样本方式，无需特定任务训练。由于缺乏可靠材质标注数据，我们使用DeepEval中的LLM-as-a-Judge进行评估。在Fusion/ABS与ShapeNet共1000个形状上，方法展现出高语义与材质合理性。结果表明，语言模型可作为通用先验，有效连接3D几何推理与材质理解。
原文摘要 · Abstract (English)
Most existing 3D shape datasets and models focus solely on geometry, overlooking the material properties that determine how objects appear. We introduce a two-stage large language model (LLM) based method for inferring material composition directly from 3D point clouds with coarse segmentations. Our key insight is to decouple reasoning about what an object is from what it is made of. In the first stage, an LLM predicts the object's semantic; in the second stage, it assigns plausible materials to each geometric segment, conditioned on the inferred semantics. Both stages operate in a zero-shot manner, without task-specific training. Because existing datasets lack reliable material annotations, we evaluate our method using an LLM-as-a-Judge implemented in DeepEval. Across 1,000 shapes from Fusion/ABS and ShapeNet, our method achieves high semantic and material plausibility. These results demonstrate that language models can serve as general-purpose priors for bridging geometric reasoning and material understanding in 3D data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。