构建细粒度相似物体场景，评估3D模型在复杂环境下的理解能力。
ObjVariantEnsemble: Advancing Point Cloud LLM Evaluation in Challenging Scenes with Subtly Distinguished Objects
- 设计可系统生成相似物体的场景，增强评测挑战性。
- 引入LLM-VLM协作标注，精准捕捉细微差异特征。
- 适合研究3D理解与具身智能的开发者参考。
3D场景理解是重要任务，近期研究聚焦于将点云3D表示与文本对齐以支持具身智能。然而，由于缺乏全面的3D基准测试，现有模型在真实场景（尤其是存在细微差别的相似物体）中的表现仍不充分。为推动更全面的3D模型评估，本文提出ObjVariantEnsemble方案，系统生成包含特定类别、颜色、形状、数量和空间关系的场景，以满足模型评估需求。更重要的是，我们有意构造具有部分相似性的物体场景，并设计基于大语言模型与视觉语言模型协同的标注器，精确捕捉关键差异作为标注。所构建的基准能更有效地挑战3D模型，揭示其在理解上的缺陷，有望推动3D模型的进一步发展。
原文摘要 · Abstract (English)
3D scene understanding is an important task, and there has been a recent surge of research interest in aligning 3D representations of point clouds with text to empower embodied AI. However, due to the lack of comprehensive 3D benchmarks, the capabilities of 3D models in real-world scenes, particularly those that are challenging with subtly distinguished objects, remain insufficiently investigated. To facilitate a more thorough evaluation of 3D models' capabilities, we propose a scheme, ObjVariantEnsemble, to systematically introduce more scenes with specified object classes, colors, shapes, quantities, and spatial relationships to meet model evaluation needs. More importantly, we intentionally construct scenes with similar objects to a certain degree and design an LLM-VLM-cooperated annotator to capture key distinctions as annotations. The resultant benchmark can better challenge 3D models, reveal their shortcomings in understanding, and potentially aid in the further development of 3D models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。