用专家分工方法精准建模复杂场景下的多变运动场。
GeoMoE: Divide-and-Conquer Motion Field Modeling with Mixture-of-Experts for Two-View Geometry
- 以概率先验分解运动场,按结构划分异质子区域。
- 引入专家增强双路径修复器,实现细粒度运动场校正。
- 适合处理视角、尺度变化大且深度不连续的复杂场景。
当前两视图几何研究越来越重视在估计图像间运动场时施加平滑性和全局一致性先验。然而,在包含极端视角与尺度变化及显著深度不连续的复杂真实场景中,运动场常呈现多样且异质的运动模式。现有方法缺乏针对性建模策略,未能显式考虑这种多样性,导致估计的运动场偏离其真实的底层结构与分布。我们观察到,混合专家(Mixture-of-Experts, MoE)可为不同运动子域分配专用专家,实现对异质运动模式的分而治之。基于此,我们提出GeoMoE,一种重构两视图几何中运动场建模的轻量级框架。首先设计概率先验引导的分解策略,利用内点概率信号对运动场进行结构感知分解,有效抑制异常值引起的偏差。其次引入MoE增强的双路径修复器,沿空间上下文与通道语义路径增强各子域,并将其路由至定制专家进行靶向建模,从而解耦异质运动模式,抑制跨子域干扰与表征纠缠,实现精细化运动场修复。该极简设计使GeoMoE在相对位姿与单应性估计上超越现有最先进方法,并展现出强泛化能力。代码与预训练模型见:https://github.com/JiajunLe/GeoMoE。
原文摘要 · Abstract (English)
Recent progress in two-view geometry increasingly emphasizes enforcing smoothness and global consistency priors when estimating motion fields between pairs of images. However, in complex real-world scenes, characterized by extreme viewpoint and scale changes as well as pronounced depth discontinuities, the motion field often exhibits diverse and heterogeneous motion patterns. Most existing methods lack targeted modeling strategies and fail to explicitly account for this variability, resulting in estimated motion fields that diverge from their true underlying structure and distribution. We observe that Mixture-of-Experts (MoE) can assign dedicated experts to motion sub-fields, enabling a divide-and-conquer strategy for heterogeneous motion patterns. Building on this insight, we re-architect motion field modeling in two-view geometry with GeoMoE, a streamlined framework. Specifically, we first devise a Probabilistic Prior-Guided Decomposition strategy that exploits inlier probability signals to perform a structure-aware decomposition of the motion field into heterogeneous sub-fields, sharply curbing outlier-induced bias. Next, we introduce an MoE-Enhanced Bi-Path Rectifier that enhances each sub-field along spatial-context and channel-semantic paths and routes it to a customized expert for targeted modeling, thereby decoupling heterogeneous motion regimes, suppressing cross-sub-field interference and representational entanglement, and yielding fine-grained motion-field rectification. With this minimalist design, GeoMoE outperforms prior state-of-the-art methods in relative pose and homography estimation and shows strong generalization. The source code and pre-trained models are available at https://github.com/JiajunLe/GeoMoE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。