无需CAD模型,仅用3D扫描就能提升工业物体识别准确率。
Where Appearance Fails, Geometry Recognizes: A CAD-Free 3D Shape Prior That Complements Vision Foundation Models

- 用RGB-D扫描重建3D形状,生成类级原型,融合图像特征。
- 在纹理少的工业零件上识别率从56%提升至59%,显著优于纯图像方法。
- 适用于缺乏标签和纹理的场景,尤其适合机器人视觉应用。
在制造与服务机器人中,对未标注物体进行识别是一个常见挑战,但传统依赖计算机辅助设计(CAD)模型的先验信息往往不可用。二维图像无法提供形状先验,而冻结的视觉基础模型在几何相似、低纹理工业部件上表现不佳。本文研究短时物体中心扫描能带来什么额外识别价值:通过3D高斯泼溅(3DGS)重建每个物体,生成类级形状原型,并与冻结的DINOv2图像特征融合。实验表明,仅使用深度信息(在T-LESS数据集上)、3DGS或CAD重建所得几何信息,其识别性能与原始CAD相当(在HOPE上持平,在T-LESS上差距≤1.6点),且3DGS仅是获取点云的一种便捷路径。在形状差异明显的家用物品(HOPE)上,仅用几何信息达到0.920准确率,高于图像仅有的0.832;固定权重融合后为0.872,低于天花板。在形状易混淆的无纹理工业部件(T-LESS)上,融合后准确率达0.591,高于单信号(0.560)。该先验具有互补性:挽救更多图像失败案例,且在部分遮挡下优势更明显。关键在于几何结构本身,而非渲染像素——3DGS渲染图对图像侧无帮助,且冻结特征识别几乎不受光照影响(<2.5点波动)。研究聚焦于识别任务,不涉及BOP姿态基准。
原文摘要 · Abstract (English)
Recognizing specific objects onboarded without a labeled training set recurs across manufacturing and service robotics, yet the conventional renderable prior, a computer-aided-design (CAD) model, is often unavailable. Two-dimensional capture supplies no shape prior, and frozen foundation features fail on geometrically similar, low-texture industrial parts. We ask what a short object-centric scan buys for recognition beyond the captured images themselves: each object is reconstructed with 3D Gaussian Splatting (3DGS), summarized into a per-class shape prototype, and fused with frozen DINOv2 image features. First, the scan recovers the recognition value of CAD without CAD: geometry from RGB-D depth (on T-LESS), 3DGS, and CAD gives comparable recognition (tied on HOPE, within 1.6 points on T-LESS); 3DGS is only a convenient route to a point cloud. Second, the payoff is governed by how recognizable the shape is: on shape-distinctive household objects (HOPE) geometry alone reaches 0.920 versus image-only 0.832, a ceiling below which fixed-weight fusion (0.872) sits. On shape-confusable textureless industrial parts (T-LESS) the gain is modest but consistent (0.560 to 0.591 fused, above both single signals). Third, the prior is complementary, not uniformly additive: it rescues far more image failures than it breaks successes, and its benefit grows under partial occlusion. Finally, the worth lies in geometry, not rendered pixels: 3DGS renderings do not help the image side, and frozen-feature recognition is nearly lighting-invariant (within 2.5 points). The study is scoped to recognition, not the BOP pose benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。