arXiv:2411.00164cs.CV2024-11被引 3

提出几何感知的3D网格Transformer,解决无序与尺寸变化难题。

A Recipe for Geometry-Aware 3D Mesh Transformers

  • 基于代数多重网格的谱保持分块方法,生成稳定拓扑嵌入。
  • 引入测地掩码与跨注意力交互,提升分割与分类精度。
  • 适合需要几何不变性的3D形状分析任务,如语义分割。

将基于块的Transformer应用于非结构化几何数据(如多边形网格)面临重大挑战,主要源于缺乏标准排序和输入尺寸不一。现有方法或依赖计算量大的节点级标记处理大对象,或通过重采样统一块大小。此外,这些方法通常缺乏几何感知且稳定的结构嵌入(SE),常采用简单的绝对位置编码(如3D坐标),损害了保距不变性,影响语义分割等任务表现。本文系统考察了几何感知3D网格Transformer的各个组件,从标记化到结构编码。首先提出基于代数多重网格方法的谱保持标记化;其次设计支持变节点数量块的特征嵌入策略。与仅使用点级MLP的基线模型对比表明:1)热扩散生成的结构与位置嵌入对3D网格Transformer至关重要;2)测地掩码与跨注意力特征交互显著增强学习效果;3)所提方法在挑战性的分割与分类任务中表现出更优性能与效率。

原文摘要 · Abstract (English)

Utilizing patch-based transformers for unstructured geometric data such as polygon meshes presents significant challenges, primarily due to the absence of a canonical ordering and variations in input sizes. Prior approaches to handling 3D meshes and point clouds have either relied on computationally intensive node-level tokens for large objects or resorted to resampling to standardize patch size. Moreover, these methods generally lack a geometry-aware, stable Structural Embedding (SE), often depending on simplistic absolute SEs such as 3D coordinates, which compromise isometry invariance essential for tasks like semantic segmentation. In our study, we meticulously examine the various components of a geometry-aware 3D mesh transformer, from tokenization to structural encoding, assessing the contribution of each. Initially, we introduce a spectral-preserving tokenization rooted in algebraic multigrid methods. Subsequently, we detail an approach for embedding features at the patch level, accommodating patches with variable node counts. Through comparative analyses against a baseline model employing simple point-wise Multi-Layer Perceptrons (MLP), our research highlights critical insights: 1) the importance of structural and positional embeddings facilitated by heat diffusion in general 3D mesh transformers; 2) the effectiveness of novel components such as geodesic masking and feature interaction via cross-attention in enhancing learning; and 3) the superior performance and efficiency of our proposed methods in challenging segmentation and classification tasks.

3D网格Transformer几何感知结构嵌入

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。