无需标注数据,自动拼接图像且更自然真实。
Robust Image Stitching with Optimal Plane
- 双分支结构融合语义不变特征与细粒度特征。
- 引入虚拟最优平面,显著提升拼接鲁棒性与自然度。
- 适合复杂场景下图像拼接,尤其对无标注数据有效。
我们提出RopStitch,一种无监督深度图像拼接框架,兼具鲁棒性与自然性。为增强鲁棒性,采用双分支架构,分别提取语义不变的预训练特征与可学习的细粒度判别特征,并在相关性层面通过可控因子融合。针对内容对齐与结构保持之间的矛盾,提出虚拟最优平面概念,将问题建模为单应性分解系数估计,设计迭代系数预测器与最小语义失真约束以识别最优平面。最终通过双向投影至最优平面实现拼接。大量实验表明,RopStitch在多个数据集上显著优于现有方法,尤其在场景鲁棒性与内容自然性方面表现突出。
原文摘要 · Abstract (English)
We present \textit{RopStitch}, an unsupervised deep image stitching framework with both robustness and naturalness. To ensure the robustness of \textit{RopStitch}, we propose to incorporate the universal prior of content perception into the image stitching model by a dual-branch architecture. It separately captures coarse and fine features and integrates them to achieve highly generalizable performance across diverse unseen real-world scenes. Concretely, the dual-branch model consists of a pretrained branch to capture semantically invariant representations and a learnable branch to extract fine-grained discriminative features, which are then merged into a whole by a controllable factor at the correlation level. Besides, considering that content alignment and structural preservation are often contradictory to each other, we propose a concept of virtual optimal planes to relieve this conflict. To this end, we model this problem as a process of estimating homography decomposition coefficients, and design an iterative coefficient predictor and minimal semantic distortion constraint to identify the optimal plane. This scheme is finally incorporated into \textit{RopStitch} by warping both views onto the optimal plane bidirectionally. Extensive experiments across various datasets demonstrate that \textit{RopStitch} significantly outperforms existing methods, particularly in scene robustness and content naturalness. The code is available at {\color{red}https://github.com/MmelodYy/RopStitch}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。