用高质量数据筛选和渐进训练,让3D场景理解更高效
DC-Scene: Data-Centric Learning for 3D Scene Understanding
- 基于CLIP的双指标过滤器,自动挑出优质3D场景图文对
- 仅用75%高质量样本,性能达全量数据的99.8%(CIDEr 86.1)
- 适合资源有限但追求高精度3D理解的研究者和工程师
3D场景理解在机器人、自动驾驶和增强现实等视觉应用中至关重要。然而,学习型3D场景理解仍面临两大挑战:一是3D场景规模大、结构复杂,导致计算成本高、训练慢;二是高质量标注的3D数据集远少于2D视觉可用数据。为此,本文提出面向3D场景理解的数据中心框架DC-Scene,强调提升数据质量与训练效率。具体地,引入基于CLIP的双指标质量(DIQ)过滤器,结合视觉-语言对齐分数与文本困惑度,辅以渐进式课程调度器,从顶部25%逐步扩展至75%的场景-描述对进行训练。该策略有效剔除噪声样本,显著降低对大规模标注3D数据的依赖。在ScanRefer和Nr3D数据集上的大量实验表明,DC-Scene以约三分之二的训练成本,实现最先进性能(使用前75%子集时CIDEr为86.1,而全数据集为85.4),验证了精炼高质量样本可超越全面训练。代码将发布于https://github.com/AIGeeksGroup/DC-Scene。
原文摘要 · Abstract (English)
3D scene understanding plays a fundamental role in vision applications such as robotics, autonomous driving, and augmented reality. However, advancing learning-based 3D scene understanding remains challenging due to two key limitations: (1) the large scale and complexity of 3D scenes lead to higher computational costs and slower training compared to 2D counterparts; and (2) high-quality annotated 3D datasets are significantly scarcer than those available for 2D vision. These challenges underscore the need for more efficient learning paradigms. In this work, we propose DC-Scene, a data-centric framework tailored for 3D scene understanding, which emphasizes enhancing data quality and training efficiency. Specifically, we introduce a CLIP-driven dual-indicator quality (DIQ) filter, combining vision-language alignment scores with caption-loss perplexity, along with a curriculum scheduler that progressively expands the training pool from the top 25% to 75% of scene-caption pairs. This strategy filters out noisy samples and significantly reduces dependence on large-scale labeled 3D data. Extensive experiments on ScanRefer and Nr3D demonstrate that DC-Scene achieves state-of-the-art performance (86.1 CIDEr with the top-75% subset vs. 85.4 with the full dataset) while reducing training cost by approximately two-thirds, confirming that a compact set of high-quality samples can outperform exhaustive training. Code will be available at https://github.com/AIGeeksGroup/DC-Scene.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。