RD-ViT用循环深度结构减少分割模型对数据的依赖，提升效率与性能。
RD-ViT: Recurrent-Depth Vision Transformer for Semantic Segmentation with Reduced Data Dependence Extending the Recurrent-Depth Transformer Architecture to Dense Prediction
- 用单个共享块循环替代多层独立Transformer，降低参数量和数据需求
- 2D/3D测试中，仅用10%数据即达标准ViT精度，3D模型参数减半仍保持99.4%性能
- 自适应计算时间与专家路由机制让模型自动聚焦关键区域，适合医疗图像分割
视觉变压器（ViTs）在分割任务中表现卓越，但需大量训练数据，因每层参数独立学习。本文提出RD-ViT，将循环深度变压器（RDT）架构扩展至密集预测任务，支持2D与3D输入。通过将深层独立Transformer块替换为单一共享块循环T次，并引入LTI稳定状态注入、自适应计算时间（ACT）、深度LoRA适配及可选混合专家（MoE）前馈网络，实现高效建模。在ACDC心脏MRI分割基准上进行纯真实实验（基于Google Colab）。2D场景下，10%数据时Dice达0.774（标准ViT为0.762），全量数据时达0.882（标准为0.872）。3D场景中，含MoE的RD-ViT仅3.0M参数，Dice达0.812，达到标准ViT性能（0.817）的99.4%，参数仅为53%。专家利用分析显示，各专家自发专精于不同心腔（RV、MYO、LV）；ACT停止图表明边界区域计算分配更高，平均推理迭代数从2.6降至1.4，体现计算效率提升。深度外推使推理循环数超过训练无性能下降。所有代码、笔记与结果均已公开。
原文摘要 · Abstract (English)
Vision Transformers (ViTs) achieve state-of-the-art segmentation accuracy but require large training datasets because each layer has unique parameters that must be learned independently. We present RD-ViT, a Recurrent-Depth Vision Transformer that adapts the Recurrent-Depth Transformer (RDT) architecture to dense prediction tasks, supporting both 2D and 3D inputs. RD-ViT replaces the deep stack of unique transformer blocks with a single shared block looped T times, augmented with LTI-stable state injection for guaranteed convergence, Adaptive Computation Time (ACT) for spatial compute allocation, depth-wise LoRA adaptation, and optional Mixture-of-Experts (MoE) feed-forward networks for category-specific specialization. We evaluate on the ACDC cardiac MRI segmentation benchmark in both 2D slice-level and 3D volumetric settings with exclusively real experiments executed in Google Colab. In 2D, RD-ViT outperforms standard ViT at 10% training data (Dice 0.774 vs 0.762) and at full data (0.882 vs 0.872). In 3D, RD-ViT with MoE achieves Dice 0.812 with 3.0M parameters, reaching 99.4% of standard ViT performance (0.817) at 53% of the parameter count. MoE expert utilization analysis reveals that different experts spontaneously specialize for different cardiac structures (RV, MYO, LV) without explicit routing supervision. ACT halting maps show higher compute allocation at cardiac boundaries, and the mean ponder time decreases from 2.6 to 1.4 iterations during training, demonstrating learned computational efficiency. Depth extrapolation enables inference with more loops than training without degradation. All code, notebooks, and results are publicly released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。