无需标注数据，通过分治策略实现跨模态图像拼接
SSHNet: Unsupervised Cross-modal Homography Estimation via Problem Reformulation and Split Optimization
- 将难题拆解为两个可独立训练的子任务，分别处理几何变换与模态转换
- 在OPT-SAR数据集上相较以往无监督方法降低47.4%~85.8%误差
- 适合需要跨域图像对齐且无标注数据的场景，如遥感与红外图像融合
本文提出一种新的无监督跨模态单应性估计学习框架——SSHNet。该框架将无监督跨模态单应性估计问题重新构建为两个有监督子问题，分别由专用网络解决：单应性估计网络和模态转换网络。为保证训练稳定性，引入分步优化策略，分别训练各子问题中的网络。同时，设计额外的单应性特征空间监督机制以增强特征一致性，进一步提升估计精度。此外，采用简单有效的知识蒸馏训练技术，在保持性能的同时减少模型参数并增强跨域泛化能力。SSHNet的训练稳定性使其可与多种单应性估计架构协同使用。实验表明，当以IHN作为单应性估计网络时（即SSHNet-IHN），在挑战性的OPT-SAR数据集上显著优于以往无监督方法，相比监督方法MHN和LocalTrans分别实现47.4%和85.8%的平均角点误差（MACE）降低。
原文摘要 · Abstract (English)
We propose a novel unsupervised cross-modal homography estimation learning framework, named Split Supervised Homography estimation Network (SSHNet). SSHNet reformulates the unsupervised cross-modal homography estimation into two supervised sub-problems, each addressed by its specialized network: a homography estimation network and a modality transfer network. To realize stable training, we introduce an effective split optimization strategy to train each network separately within its respective sub-problem. We also formulate an extra homography feature space supervision to enhance feature consistency, further boosting the estimation accuracy. Moreover, we employ a simple yet effective distillation training technique to reduce model parameters and improve cross-domain generalization ability while maintaining comparable performance. The training stability of SSHNet enables its cooperation with various homography estimation architectures. Experiments reveal that the SSHNet using IHN as homography estimation network, namely SSHNet-IHN, outperforms previous unsupervised approaches by a significant margin. Even compared to supervised approaches MHN and LocalTrans, SSHNet-IHN achieves 47.4% and 85.8% mean average corner errors (MACEs) reduction on the challenging OPT-SAR dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。