无需训练或修改结构，让2D模型直接具备3D理解能力
PlaneCycle: Training-Free 2D-to-3D Lifting of Foundation Models Without Adapters
- 通过循环分布空间聚合到三个正交平面实现无参数3D升维
- 在6个3D分类和3个分割任务上达到接近全量训练模型的性能
- 适用于任意2D模型，特别适合快速部署3D视觉应用
大规模2D基础模型具有强大的可迁移表征能力，但将其扩展至3D体数据通常需要重新训练、引入适配器或改变架构。我们提出PlaneCycle，一种无需训练、无需适配器、架构无关的2D到3D升维算子。PlaneCycle通过在网络深度中循环地将空间聚合分布在横向(HW)、深度-宽度(DW)和深度-高度(DH)三个正交平面上，实现渐进式3D融合，同时保留预训练模型的归纳偏置。该方法不引入额外参数，适用于任意2D网络。使用预训练的DINOv3模型，在六个3D分类和三个3D分割基准上评估，未进行任何训练的提升模型即展现出内在3D融合能力；在线性探测下，其性能优于逐片处理的2D基线模型和强3D模型，接近全量训练模型表现。全微调后，PlaneCycle性能与标准3D架构相当，证明其作为无缝2D到3D升维算子的巨大潜力。结果表明，无需结构修改或重训练即可从预训练2D基础模型中解锁3D能力。
原文摘要 · Abstract (English)
Large-scale 2D foundation models exhibit strong transferable representations, yet extending them to 3D volumetric data typically requires retraining, adapters, or architectural redesign. We introduce PlaneCycle, a training-free, adapter-free operator for architecture-agnostic 2D-to-3D lifting of foundation models. PlaneCycle reuses the original pretrained 2D backbone by cyclically distributing spatial aggregation across orthogonal HW, DW, and DH planes throughout network depth, enabling progressive 3D fusion while preserving pretrained inductive biases. The method introduces no additional parameters and is applicable to arbitrary 2D networks. Using pretrained DINOv3 models, we evaluate PlaneCycle on six 3D classification and three 3D segmentation benchmarks. Without any training, the lifted models exhibit intrinsic 3D fusion capability and, under linear probing, outperform slice-wise 2D baselines and strong 3D counterparts, approaching the performance of fully trained models. With full fine-tuning, PlaneCycle matches standard 3D architectures, highlighting its potential as a seamless and practical 2D-to-3D lifting operator. These results demonstrate that 3D capability can be unlocked from pretrained 2D foundation models without structural modification or retraining. Code is available at https://github.com/HINTLab/PlaneCycle.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。