用分层注意力和载体令牌实现3D超分辨率长距离建模
MTVNet: Mapping using Transformers for Volumes -- Network for Super-Resolution with Long-Range Interactions
- 分层多尺度注意力结构，通过载体令牌传递粗到细信息
- 在5个3D数据集上优于当前最佳模型，尤其对大图像效果更优
- 适合需要长程依赖的3D医学影像超分辨率任务
以往3D体数据超分辨率难以应用2D中先进的基于Transformer的模型，因3D自注意力机制内存开销大，限制了感受野。为此，本文提出MTVNet，一种基于分层注意力块与多尺度载体令牌的多尺度Transformer模型。该方法将粗分辨率下大区域信息逐步传递至细分辨率区域，以预测超分辨图像。每个分辨率层级使用Transformer层，使每层令牌数受限，从而实现比以往更大的感受野。我们在五个3D数据集上实验对比，结果表明该方法在增大感受野方面具有优势，尤其对超出常用3D数据集尺寸的图像表现更佳。代码已开源。
原文摘要 · Abstract (English)
Until now, it has been difficult for volumetric super-resolution to utilize the recent advances in transformer-based models seen in 2D super-resolution. The memory required for self-attention in 3D volumes limits the receptive field. Therefore, long-range interactions are not used in 3D to the extent done in 2D and the strength of transformers is not realized. We propose a multi-scale transformer-based model based on hierarchical attention blocks combined with carrier tokens at multiple scales to overcome this. Here information from larger regions at coarse resolution is sequentially carried on to finer-resolution regions to predict the super-resolved image. Using transformer layers at each resolution, our coarse-to-fine modeling limits the number of tokens at each scale and enables attention over larger regions than what has previously been possible. We experimentally compare our method, MTVNet, against state-of-the-art volumetric super-resolution models on five 3D datasets demonstrating the advantage of an increased receptive field. This advantage is especially pronounced for images that are larger than what is seen in popularly used 3D datasets. Our code is available at https://github.com/AugustHoeg/MTVNet
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。