arXiv:2511.22429cs.CV2025-11NeurIPS被引 2

用单目教师模型蒸馏细节,让3D重建模型更清晰准确。

Fin3R: Fine-tuning Feed-forward 3D Reconstruction Models via Monocular Knowledge Distillation

  • 冻结解码器,仅微调图像编码器,提升几何细节提取能力。
  • 在多个模型上实现边界更锐利、结构更完整,精度显著提升。
  • 只需轻量级适配器,推理成本几乎不变,适合部署应用。

我们提出Fin3R,一种简单有效的前馈3D重建模型微调方法。前馈重建模型在一次前向传播中将多视角输入图像的点云回归到参考坐标系,并输出辅助信息。然而,现有模型因高保真深度与位姿监督稀缺、多视角点云回归固有的几何错位问题,难以捕捉精细几何结构且鲁棒性不足。Fin3R通过额外轻量微调步骤联合解决上述问题:冻结负责视图匹配的解码器,仅微调专注于特征提取的图像编码器;利用定制轻量级LoRA适配器,从强单目教师模型中蒸馏大规模无标签数据上的几何细节。我们在DUSt3R、MASt3R、CUT3R和VGGT等多类模型上验证该方法,微调后模型在单视图与多视图设置下均呈现更锐利的边界、更复杂的结构恢复能力,且几何精度更高。仅增加极小的LoRA参数,测试时内存与延迟几乎不受影响。

原文摘要 · Abstract (English)

We present Fin3R, a simple, effective, and general fine-tuning method for feed-forward 3D reconstruction models. The family of feed-forward reconstruction model regresses pointmap of all input images to a reference frame coordinate system, along with other auxiliary outputs, in a single forward pass. However, we find that current models struggle with fine geometry and robustness due to (\textit{i}) the scarcity of high-fidelity depth and pose supervision and (\textit{ii}) the inherent geometric misalignment from multi-view pointmap regression. Fin3R jointly tackles two issues with an extra lightweight fine-tuning step. We freeze the decoder, which handles view matching, and fine-tune only the image encoder-the component dedicated to feature extraction. The encoder is enriched with fine geometric details distilled from a strong monocular teacher model on large, unlabeled datasets, using a custom, lightweight LoRA adapter. We validate our method on a wide range of models, including DUSt3R, MASt3R, CUT3R, and VGGT. The fine-tuned models consistently deliver sharper boundaries, recover complex structures, and achieve higher geometric accuracy in both single- and multi-view settings, while adding only the tiny LoRA weights, which leave test-time memory and latency virtually unchanged. Project page: \href{http://visual-ai.github.io/fin3r}{https://visual-ai.github.io/fin3r}

3D重建知识蒸馏轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。