让3D重建模型在测试时自动适应新场景,无需重新训练。
MVS-TTA: Test-Time Adaptation for Multi-View Stereo via Meta-Auxiliary Learning
- 用跨视角一致性损失做辅助任务,指导测试时模型自我调整。
- 在DTU和BlendedMVS上提升重建精度,跨数据集效果也稳定。
- 不依赖特定模型,适配性强,适合各类学习型三维重建方法。
基于学习的多视图立体(MVS)方法依赖大规模数据训练,虽有显著进展,但因训练数据分布有限,泛化能力不足。而基于优化的方法虽可实现场景自适应,却难以扩展且需逐场景耗时优化。本文提出MVS-TTA,一种高效的测试时自适应(TTA)框架,通过元辅助学习策略,使学习型MVS模型在推理时能借助自监督的跨视角一致性损失进行动态调整。该方法无需修改主模型结构,具备模型无关性,可在多种主流MVS方法上轻松集成。在标准数据集(DTU、BlendedMVS)及跨数据集泛化设置下,实验表明其显著提升性能,甚至对顶尖模型也有增益。据我们所知,这是首个将基于优化的测试时自适应与元学习结合应用于学习型MVS的工作。
原文摘要 · Abstract (English)
Recent learning-based multi-view stereo (MVS) methods are data-driven and have achieved remarkable progress due to large-scale training data and advanced architectures. However, their generalization remains sub-optimal due to fixed model parameters trained on limited training data distributions. In contrast, optimization-based methods enable scene-specific adaptation but lack scalability and require costly per-scene optimization. In this paper, we propose MVS-TTA, an efficient test-time adaptation (TTA) framework that enhances the adaptability of learning-based MVS methods by bridging these two paradigms. Specifically, MVS-TTA employs a self-supervised, cross-view consistency loss as an auxiliary task to guide inference-time adaptation. We introduce a meta-auxiliary learning strategy to train the model to benefit from auxiliary-task-based updates explicitly. Our framework is model-agnostic and can be applied to a wide range of MVS methods with minimal architectural changes. Extensive experiments on standard datasets (DTU, BlendedMVS) and a challenging cross-dataset generalization setting demonstrate that MVS-TTA consistently improves performance, even when applied to state-of-the-art MVS models. To our knowledge, this is the first attempt to integrate optimization-based test-time adaptation into learning-based MVS using meta-learning. The code will be available at https://github.com/mart87987-svg/MVS-TTA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。