arXiv:2603.10568cs.CV2026-03被引 1

统一语义与几何特征,提升图像拼接精度。

UniStitch: Unifying Semantic and Geometric Features for Image Stitching

  • 用神经点变换器将稀疏关键点转为稠密语义图
  • 自适应专家融合模块提升复杂场景拼接效果
  • 适合需要高精度拼接的视觉应用开发者

传统图像拼接方法依赖手工设计的几何特征估计形变,而近年基于学习的方法则使用神经网络提取语义特征。两者发展路径分离,迄今几乎无融合。本文提出UniStitch,首个统一多模态特征的图像拼接框架。为对齐离散几何特征(如关键点)与连续语义特征图,提出神经点变换器(NPT)模块,将无序稀疏的一维关键点转换为有序稠密的二维语义图。为进一步融合两类表示的优势,设计自适应专家混合(AMoE)模块,在融合过程中动态聚焦于更可靠的特征,有效应对任一模态受损的复杂场景。融合表示可嵌入通用深度拼接流程,显著优于单一特征方法。实验表明,UniStitch大幅超越现有最先进方法,为传统与学习型拼接技术的统一开辟新路径。

原文摘要 · Abstract (English)

Traditional image stitching methods estimate warps from hand-crafted geometric features, whereas recent learning-based solutions leverage semantic features from neural networks instead. These two lines of research have largely diverged along separate evolution, with virtually no meaningful convergence to date. In this paper, we take a pioneering step to bridge this gap by unifying semantic and geometric features with UniStitch, a unified image stitching framework from multimodal features. To align discrete geometric features (i.e., keypoint) with continuous semantic feature maps, we present a Neural Point Transformer (NPT) module, which transforms unordered, sparse 1D geometric keypoints into ordered, dense 2D semantic maps. Then, to integrate the advantages of both representations, an Adaptive Mixture of Experts (AMoE) module is designed to fuse geometric and semantic representations. It dynamically shifts focus toward more reliable features during the fusion process, allowing the model to handle complex scenes, especially when either modality might be compromised. The fused representation can be adopted into common deep stitching pipelines, delivering significant performance gains over any single feature. Experiments show that UniStitch outperforms existing state-of-the-art methods with a large margin, paving the way for a unified paradigm between traditional and learning-based image stitching.

图像拼接多模态融合深度学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。