arXiv:2603.05999cs.CV2026-03中稿 · ECCV被引 1

用自调制机制让全景深度估计更好继承透视预训练模型优势。

RePer-360: Releasing Perspective Priors for 360$^\circ$ Depth Estimation via Self-Modulation

  • 通过双投影互补引导,轻量级调制全景特征分布。
  • 仅用1%数据即超越标准微调,RMSE降低约20%。
  • 适合想高效迁移透视模型到全景场景的研究者。

近期基于透视图像训练的深度基础模型表现优异,但在全景图像上泛化能力差,因透视与全景间存在显著几何差异。完全微调通常需大量全景数据。为此,我们提出RePer-360,一种畸变感知的自调制框架,用于单目全景深度估计,可在保留强大预训练透视先验的前提下适配模型。具体地,设计轻量级几何对齐引导模块,从等距矩形投影(ERP)和立方体贴图(CP)两种互补投影中提取调制信号,指导模型向全景域迁移而不覆盖原有知识。引入自条件AdaLN-Zero机制,生成像素级缩放因子,缩小透视与全景特征分布差距。此外,引入立方体贴图域一致性损失,提升训练稳定性和跨投影对齐。相比传统微调方法,仅使用1%训练数据即实现性能超越;在同域训练下,RMSE进一步降低约20%。代码已开源。

原文摘要 · Abstract (English)

Recent depth foundation models trained on perspective imagery achieve strong performance, yet generalize poorly to 360$^\circ$ images due to the substantial geometric discrepancy between perspective and panoramic domains. Moreover, fully fine-tuning these models typically requires large amounts of panoramic data. To address this issue, we propose RePer-360, a distortion-aware self-modulation framework for monocular panoramic depth estimation that adapts depth foundation models while preserving powerful pretrained perspective priors. Specifically, we design a lightweight geometry-aligned guidance module to derive a modulation signal from two complementary projections (i.e., ERP and CP) and use it to guide the model toward the panoramic domain without overwriting its pretrained perspective knowledge. We further introduce a Self-Conditioned AdaLN-Zero mechanism that produces pixel-wise scaling factors to reduce the feature distribution gap between the perspective and panoramic domains. In addition, a cubemap-domain consistency loss further improves training stability and cross-projection alignment. By shifting the focus from complementary-projection fusion to panoramic domain adaptation under preserved pretrained perspective priors, RePer-360 surpasses standard fine-tuning methods while using only 1\% of the training data. Under the same in-domain training setting, it further achieves an approximately 20\% improvement in RMSE. The code is available at https://github.com/munimo/RePer360.

深度估计全景视觉自调制迁移学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。