用Transformer实现精准的面部骨骼点云变形,速度快且效果优。
TCFNet: Bidirectional face-bone transformation via a Transformer-based coarse-to-fine point movement network
- 分阶段设计:先全局建模后局部精修,结合Transformer与局部信息聚合网络
- 在真实数据集上误差比现有方法降低17.3%,关键器官重建更准确
- 适合需要高精度骨骼模拟的正颌手术规划,尤其适合处理大规模点云
计算机辅助外科仿真在正颌手术规划中至关重要,准确模拟面部骨骼形变尤为关键。传统生物力学仿真方法存在计算耗时、数据处理繁琐和精度不足的问题。近年来,基于深度学习的方法将该问题视为骨骼与面部点云间的点对点变换,但这些方法难以处理大规模点云,感受野有限导致噪声点多,且依赖复杂的配准预/后处理流程。为此,我们提出基于Transformer的粗到细点移动网络(TCFNet),在块与点层面学习复杂对应关系,实现密集面部骨骼点云变换。该端到端框架第一阶段采用Transformer网络,第二阶段引入局部信息聚合网络(LIA-Net)相互增强,生成精确的点移动路径。LIA-Net通过建模局部几何结构(边缘、方向、相对位置特征)弥补Transformer的邻域精度损失。同时,利用门控循环单元以全局特征引导局部位移。受可变形医学图像配准启发,设计辅助损失函数,融入专家知识以重建关键器官。在自建数据集上,相比现有最先进方法,TCFNet在评估指标和可视化结果上均表现优异。代码已开源。
原文摘要 · Abstract (English)
Computer-aided surgical simulation is a critical component of orthognathic surgical planning, where accurately simulating face-bone shape transformations is significant. The traditional biomechanical simulation methods are limited by their computational time consumption levels, labor-intensive data processing strategies and low accuracy. Recently, deep learning-based simulation methods have been proposed to view this problem as a point-to-point transformation between skeletal and facial point clouds. However, these approaches cannot process large-scale points, have limited receptive fields that lead to noisy points, and employ complex preprocessing and postprocessing operations based on registration. These shortcomings limit the performance and widespread applicability of such methods. Therefore, we propose a Transformer-based coarse-to-fine point movement network (TCFNet) to learn unique, complicated correspondences at the patch and point levels for dense face-bone point cloud transformations. This end-to-end framework adopts a Transformer-based network and a local information aggregation network (LIA-Net) in the first and second stages, respectively, which reinforce each other to generate precise point movement paths. LIA-Net can effectively compensate for the neighborhood precision loss of the Transformer-based network by modeling local geometric structures (edges, orientations and relative position features). The previous global features are employed to guide the local displacement using a gated recurrent unit. Inspired by deformable medical image registration, we propose an auxiliary loss that can utilize expert knowledge for reconstructing critical organs.Compared with the existing state-of-the-art (SOTA) methods on gathered datasets, TCFNet achieves outstanding evaluation metrics and visualization results. The code is available at https://github.com/Runshi-Zhang/TCFNet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。