用单个工作站训练出可高效部署的3D模型,让普通实验室也能玩转3D视觉。
Distill3R: A Pipeline for Democratizing 3D Foundation Models on Commodity Hardware
- 将大模型的几何推理能力蒸馏到小模型,支持单机训练。
- 学生模型参数减少9倍,推理速度提升5倍,3天内完成训练。
- 适合无高性能算力的实验室快速定制领域专用3D模型。
尽管多视角3D重建正向大规模基础模型发展,能生成全局一致几何结构,但其训练依赖大型计算集群,对多数学术实验室构成高门槛。为弥合算力鸿沟,我们提出Distill3R框架,将3D基础模型的几何推理能力蒸馏至可在单个工作站上训练的紧凑学生模型。方法核心包括:(1) 离线缓存管道,通过压缩监督信号解耦教师模型的重负载推理与训练循环;(2) 基于置信度的蒸馏损失,利用教师不确定性实现商品硬件上的训练。我们设计了一个7200万参数的学生模型,相比6.5亿参数的教师模型,参数量减少9倍,推理速度提升5倍。学生模型可在单个工作站上3天内完成训练,而教师模型需大规模GPU集群运行长达一周。实验表明,学生模型保持了功能化3D感知所需的结构一致性和几何理解能力。通过提供可复现的单机训练方案,Distill3R为算力有限的实验室提供了低成本、易访问的研究起点,推动3D视觉研究的普惠化与边缘高效部署。本工作不追求超越当前顶尖模型,而是为无大规模算力资源的团队提供可定制化训练的基础平台。
原文摘要 · Abstract (English)
While multi-view 3D reconstruction has shifted toward large-scale foundation models capable of inferring globally consistent geometry, their reliance on massive computational clusters for training has created a significant barrier to entry for most academic laboratories. To bridge this compute divide, we introduce Distill3R, a framework designed to distill the geometric reasoning of 3D foundation models into compact students fully trainable on a single workstation. Our methodology centers on two primary innovations: (1) an offline caching pipeline that decouples heavy teacher inference from the training loop through compressed supervision signals, and (2) a confidence-aware distillation loss that leverages teacher uncertainty to enable training on commodity hardware. We propose a 72M-parameter student model which achieves a 9x reduction in parameters and a 5x inference speedup compared to its 650M-parameter teacher. The student is fully trainable in under 3 days on a single workstation, whereas its teacher requires massive GPU clusters for up to a week. We demonstrate that the student preserves the structural consistency and qualitative geometric understanding required for functional 3D awareness. By providing a reproducible, single-workstation training recipe, Distill3R serves as an exploratory entry point for democratized 3D vision research and efficient edge deployment. This work is not intended to compete with state-of-the-art foundation models, but to provide an accessible research baseline for laboratories without access to large-scale compute to train and specialize models on their own domain-specific data at minimal cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。