用Transformer融合纹理与几何信息，提升复杂3D网格语义分割精度
Semantic Segmentation of Textured Non-manifold 3D Meshes using Transformers
- 直接处理原始像素，通过纹理分支提取面级特征并融合几何信息
- 在城市网格和文物屋顶数据集上分别达到81.9%和49.7%的mF1
- 适合需要高精度三维语义理解的文物修复与城市建模场景
纹理化3D网格同时包含几何、拓扑和外观信息，但其不规则结构给基于深度学习的语义分割带来挑战。尽管一些近期方法能在无几何约束下直接操作网格，却常忽略其中丰富的纹理信息。本文提出一种纹理感知Transformer，直接从每个网格面关联的原始像素中学习，并引入新的分层学习机制实现多尺度特征聚合。纹理分支将所有面级像素汇总为可学习的令牌，与几何描述符融合后经由多层两阶段Transformer块（TSTB）处理，支持局部与全局信息传递。我们在语义城市网格（SUM）基准和新构建的文化遗产数据集（含三角级损伤标注的纹理屋瓦）上评估模型，分别取得81.9% mF1和94.3% OA，以及49.7% mF1和72.8% OA，显著优于现有方法。
原文摘要 · Abstract (English)
Textured 3D meshes jointly represent geometry, topology, and appearance, yet their irregular structure poses significant challenges for deep-learning-based semantic segmentation. While a few recent methods operate directly on meshes without imposing geometric constraints, they typically overlook the rich textural information also provided by such meshes. We introduce a texture-aware transformer that learns directly from raw pixels associated with each mesh face, coupled with a new hierarchical learning scheme for multi-scale feature aggregation. A texture branch summarizes all face-level pixels into a learnable token, which is fused with geometrical descriptors and processed by a stack of Two-Stage Transformer Blocks (TSTB), which allow for both a local and a global information flow. We evaluate our model on the Semantic Urban Meshes (SUM) benchmark and a newly curated cultural-heritage dataset comprising textured roof tiles with triangle-level annotations for damage types. Our method achieves 81.9\% mF1 and 94.3\% OA on SUM and 49.7\% mF1 and 72.8\% OA on the new dataset, substantially outperforming existing approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。