arXiv:2511.20784cs.CV2025-11被引 1

仅用10%图像块,就能重建并分类材质表面。

One Patch is All You Need: Joint Surface Material Reconstruction and Classification from Minimal Visual Cues

  • 用部分卷积U-Net+分类头,从极稀疏输入中恢复完整表面。
  • 在真实材质数据集上实现17.55 dB的PSNR和85.10%分类准确率。
  • 适合机器人感知、虚拟仿真等视觉受限场景使用。

从稀疏视觉线索中理解材质表面对机器人、模拟和材料感知应用至关重要。然而,现有方法多依赖密集或全场景观测,在视角受限或部分可见环境下效果不佳。为此,我们提出SMARC模型,实现从极少量视觉输入中联合进行表面材质重建与分类。仅需提供图像中10%连续区域的局部信息,该模型即可还原完整的RGB表面,并同时完成材质类别识别。其架构结合了部分卷积U-Net与分类头,支持在极端观测缺失下进行空间补全与语义理解。我们在真实世界表面纹理数据集Touch and Go [16] 上对比了五种模型:卷积自编码器[17]、Vision Transformer (ViT)[13]、Masked Autoencoder (MAE)[5]、Swin Transformer[9] 和 DETR[2]。SMARC以17.55 dB的PSNR和85.10%的分类准确率取得当前最优表现。结果表明,部分卷积在缺失数据下的空间推理中具有显著优势,为极简视觉下的表面理解奠定了坚实基础。

原文摘要 · Abstract (English)

Understanding material surfaces from sparse visual cues is critical for applications in robotics, simulation, and material perception. However, most existing methods rely on dense or full-scene observations, limiting their effectiveness in constrained or partial view environment. To address this challenge, we introduce SMARC, a unified model for Surface MAterial Reconstruction and Classification from minimal visual input. By giving only a single 10% contiguous patch of the image, SMARC recognizes and reconstructs the full RGB surface while simultaneously classifying the material category. Our architecture combines a Partial Convolutional U-Net with a classification head, enabling both spatial inpainting and semantic understanding under extreme observation sparsity. We compared SMARC against five models including convolutional autoencoders [17], Vision Transformer (ViT) [13], Masked Autoencoder (MAE) [5], Swin Transformer [9], and DETR [2] using Touch and Go dataset [16] of real-world surface textures. SMARC achieves state-of-the-art results with a PSNR of 17.55 dB and a material classification accuracy of 85.10%. Our findings highlight the advantages of partial convolution in spatial reasoning under missing data and establish a strong foundation for minimal-vision surface understanding.

材质识别图像补全稀疏视觉

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。